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Abstract 

This  thesis  deals  with  the  analysis  of  random  data.  Two  approaches  are 
discussed.  The  first  approach  is  a  Goodness  of  Fit  test  to  determine  whether 
or  not  random  data  samples  are  statistically  consistent  with  a  prespecified 
probability  distribution.  The  well-known  Kolmogorov- Smirnov  test,  Chi- 
Square  test,  Q-Q  Plots  and  P-P  plots  are  reviewed  and  illustrated  by  means 
of  several  examples.  A  new  algorithm,  the  Ozturk  Algorithm,  is  introduced. 

The  second  approach  deals  with  approximation  of  the  underlying  prob¬ 
ability  density  function  of  random  data  samples.  The  previously  mentioned 
well-known  tests  are  not  suitable  for  this  task.  However,  the  Ozturk  Al¬ 
gorithm  provides  a  pow-erful  solution  for  this  problem  with  a  nice  graphical 
interpretation.  Finally,  computer  simulated  results  obtained  with  the  Ozturk 
Algorithm  are  presented  and  discussed. 
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Chapter  1 


Literature  Review 

1.1  Introduction 

In  the  analysis  of  random  data,  we  encounter  situations  where  there  may  be  various 
statistical  models  or  “hypotheses”  that  need  to  be  checked  against  the  data.  The 
usual  situation  is  that  one  has  a  particular  probability  distribution  in  mind  to  be 
tested  or  checked  for  consistency  in  representing  data  from  a  certain  experiment.  The 
hypothesis  that  this  distribution  is  the  right  one  is  called  the  null  hypothesis,  often, 
denoted  by  Hq.  This  hypothesis  may  have  emerged  from  long  experience  associated 
with  an  experiment  and  it  is  desired  to  see  whether  the  hypothesis  is  still  correct  when 
there  has  been  some  change  in  circumstances  that  call  it  into  question.  Alternatively, 
the  hypothesis  may  be  the  result  of  a  theoretical  analysis  or  a  logical  argument  and 
the  theory  needs  to  be  verified. 

A  null  hypothesis  is  ordinarily  taken  to  be  quite  specific.  In  particular,  location, 
scale  and  shape  parameters  associated  wdth  the  probability  density  function  are  speci¬ 
fied  along  with  the  type  of  distribution.  For  example,  the  probability  density  function 
of  the  Weibull  distribution  changes  as  its  shape  parameter  is  changed.  Therefore, 
Weibull  distribution  having  a  different  shape  parameter  from  the  Weibull  distribu¬ 
tion  of  the  null  hypothesis  are  assumed  to  be  different.  All  the  other  distributions, 
taken  together,  define  what  is  referred  to  as  the  alternative  hypothesis,  denoted  by 
Hi-  Therefore,  another  question  that  arises  in  the  analysis  of  random  data  is  “If 
the  null  hypothesis  is  not  true,  what  are  suitable  approximations  to  the  underlying 
distribution  of  the  data?” 

To  answer  these  questions,  several  tests  have  been  proposed  and  used.  Each  of  these 
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tests  have  their  own  strengths  and  weaknesses.  Some  may  work  well  for  a  particular 
set  of  density  functions,  but  poorly  for  others.  We  focus  our  attention  on  four  of  the 
most  frequently  used  tests  for  analyzing  random  data.  Detailed  discussions  of  these 
tests  follow. 

1.2  The  Kolmogorov- Smirnov  Test 

This  test  is  based  on  the  idea  of  a  “sample  distribution  function”,  a  statistic  that 
is  the  sample  version  of  the  population  distribution  function. 

Given  a  sample  (A"!,  X2, . . .  ,X„)  of  size  n,  the  sample  distribution  function  is  the 
cumulative  distribution  function  (cdf)  of  a  discrete  probability  density  function  where 
the  random  variable  assumes  the  values  AT,A^2,  •  •  • ,  A^n  with  probability  1/n.  Conse¬ 
quently,  the  cdf  increases  in  steps  of  size  1/n  at  each  sample  value,  rising  from  0  to 

the  left  of  the  smallest  A^,  to  1  at  the  largest  Xi. 

1.2.1  Example[13] 

Consider  a  sample  of  the  following  5  observations. 

2.22,-0.83,0.18,1.18,2.05. 

The  sample  distribution  function  is  easily  constructed  after  the  sample  values  are 
marked  on  the  x-axis;  Starting  at  height  zero  for  the  values  less  than  -0.83,  the  cdf  is 
increased  successively  by  steps  of  height  1/5  at  the  ordered  sample  value  -0.83,  0.18^ 
1.18,  2.05,  and  2.22.  The  result  is  shown  in  fig. 1.1. 

The  Kolmogorov-Smirnov  test  statistic  is  defined  as  the  maximum  absolute  vertical 
deviation  D„,  of  the  sample  distribution  function,  T’„(x),  from  the  cdf,  Fo{x),  specified 
by  the  null  hypothesis  Hq.  If  the  fit  is  good,  is  expected  to  have  a  small  value. 
On  the  other  hand,  if  the  underlying  distribution  has  a  cdf  significantly  different  from 
-^o(x),  it  is  expected  that  the  fit  will  be  poor  and  will  be  large.  Consequently,  if 
values  of  exceed  a  pre-specified  value,  called  the  acceptance  limit.  Ho  is  rejected. 
Fortunately,  the  distribution  of  the  statistic  depends  only  on  the  sample  size 
and  not  on  the  shape  of  the  distribution  being  tested.  The  distribution  of  has 
been  computed  under  the  assumption  that  the  null  hypothesis  holds.  Results  of  the 
acceptance  limits  are  given  in  table  1.1  [12]  and  [13]  for  different  sample  sizes  and  for 
various  pre-selected  values  of 
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Figure  1.1:  Sample  Distribution  function  for  example  1.1.1 

a  —  Pr(reject  Ho\Ho  true)  (1-1) 

where  q  is  called  the  significance  level.  For  large  values  of  n,  asymptotic  formulae  are 
given  for  the  acceptance  limits.  In  summary,  the  test  consists  of  the  following  steps: 

1)  Plot  F„(x)  and  Fo(x)  in  the  co-ordinate  axes. 

2)  By  inspection,  determine  the  maximum  vertical  absolute  deviation,  given  by, 

£»n="“"|F„(x)-Fo(x)i.  (1.2) 

3)  Select  a  significance  level 

a  =  Pr(reject  Ho\Ho  true).  (1-3) 

4)  Accept  Ho  if  Dn  <  K  and  reject  otherwise. 

Note  that 
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I  —  a  =  Pr(accept  Hq\Hq  true). 


(1.4) 


Let  the  cdf  of  be  denoted  by  Fd„{x\Ho).  It  follows  that 

Pr{Dn  >  K\Ho)  =  1  -  FDniK\Ho)  =  a.  (1.5) 

Consequently,  K  is  the  100  x  (1  —  a)  percentile  of  Fd^{x\Ho). 

1.2.2  Example[13] 

Let  the  null  hypothesis  Fo(x)  be  Gaussian  with  mean  =  32  and  standard  deviation 
=  1.8.  Consider  the  10  observations:  31.0,  31.4,  33.3,  33.4,  33.4,  33.5,  33.7,  34.4, 
34.9,  36.2.  The  corresponding  sample  distribution  function  F„(x),  is  sketched  in 
fig.  1.2.  along  with  the  normal  distribution  whose  mean  is  32  and  whose  standard 
deviation  is  1.8.  Assume  the  significance  level  is  chosen  to  be  cc  =  0.05.  From  fig. 1.2 
it  is  determined  that  the  maximum  deviation  D„,  between  the  two  curves  is  0.56. 
From  table  1.1  the  acceptance  limit  is  K  =  0.409.  Since  D„  >  K,  Ho  is  rejected. 

Although  the  Kolmogorov-Smirnov  test  is  found  to  perform  quite  well  even  for 
small  sample  sizes,  it  has  two  principal  disadvantages. 

1.  To  perform  the  test  it  is  necessary  to  have  a  priori  knowledge  about 
the  data  in  order  to  be  able  to  specify  meaningful  null  hypotheses. 

2.  When  the  null  distribution  is  rejected,  no  information  is  provided 
about  which  distributions  are  suitable  for  approximating  the  un¬ 
derlying  distribution  of  the  data. 

1.3  The  Chi-Square  Test 

The  chi-square  test  was  originally  developed  for  discrete  random  variables.  It  is  ap¬ 
plied  to  the  case  of  continuous  random  variables  by  making  a  discrete  approximation 
to  the  continuous  probability  density  function.  Since  the  distribution  of  the  statistic 
used  becomes  tractable  only  as  the  sample  size  becomes  infinite,  the  chi-square  test 
should  be  employed  only  for  large  sample  sizes. 

Consider  a  null  hypothesis  with  probability  density  function  fo{x)  and  distribution 
function  Fo{x),  as  shown  in  fig. 1.3.  Divide  the  x-axis  into  k  contiguous  intervals 
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Sample  size 
(n) 

0.20 

Significance  level 
0.15  1  0.10  1  0.05 

0.01 

1 

0.900 

0.925 

0.950 

0.975 

0.995 

2 

0.684 

0.726 

0.776 

0.842 

0.929 

3 

0.565 

0.597 

0.642 

0.708 

0.829 

4 

0.494 

0.525 

0.564 

0.624 

0.734 

5 

0.446 

0.474 

0.510 

0.563 

0.669 

6 

0.410 

0.436 

0.470 

0.521 

0.618 

7 

0.381 

0.405 

0.438 

0.486 

0.577 

8 

0.358 

0.381 

0.411 

0.457 

0.543 

9 

0.339 

0.360 

0.388 

0.432 

0.514 

10 

0.322 

0.342 

0.368 

0.409 

0.486 

11 

0.307 

0.326 

0.352 

0.391 

0.468 

12 

0.295 

0.313 

0.338 

0.375 

13 

0.284 

0.302 

0.325 

0.361 

0.433 

14 

0.274 

0.292 

0.314 

0.349 

0.418 

15 

0.266 

0.283 

0.304 

0.338 

0.375 

16 

0.258 

0.274 

0.295 

0.328 

0.391 

17 

0.250 

0.266 

0.286 

0.318 

0.380 

18 

0.244 

0.259 

0.278 

0.309 

0.270 

19 

0.237 

0.252 

0.272 

0.301 

0.361 

20 

0.231 

0.246 

0.264 

0.294 

0.352 

25 

0.21 

0.22 

0.24 

0.264 

0.32 

30 

0.19 

0.20 

0.22 

0.242 

0.29 

35 

0.18 

0.19 

0.21 

0.23 

0.27 

40 


50 


60 


70 


80 


90 


100 


Asymptotic  formula: 


Table  1.1:  Acceptance  limits  for  the  Kolmogorov-Smirnov  test 
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Ei,E2,  . . .  ,Ek  from  left  to  right.  Note  that  the  interval,  Ei,  consists  of  the  set  of 
points  such  that 

ai-i  <  X  <  Qi 

where  gq  =  — oo  and  =  +oo.  Consequently,  a,_i  and  ai  are  the  end  points  of  the 
interval,  Ei. 

Define  the  probabilities 


Pi  =  Pr{X  e  Ei)  =  Pr(a,_i  <  X  <  Ui-i) 


=  (Fo(at)  -  Po(«i-i));  i  =  l,2,...,k 


Observe  that  pi  is  the  area  under  /o(x)  between  x  =  a,_i  and  x  =  a,-.  Also, 


tlPi  =  1- 

1=1 

Now  consider  a  random  experiment  consisting  of  n  independent  trials.  Define 


fi  —  (number  of  outcomes  in  Ei). 


According  to  the  relative  frequency  concept. 


Pi  =  lim  — . 
n— oo  n 


Note  that 


X]/.=n-  (1-9) 

i=l 

To  test  whether  the  null  hypothesis  is  statistically  consistent  with  the  data,  the 


statistic 


2  ^ifi-  npi? 
X  =22 — IT — 


(1.10) 


is  evaluated.  The  null  hypothesis  is  rejected  when  exceeds  a  critical  level  M.  To 
determine  M,  the  significance  level 
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Figure  1.3:  The  Distribution  function  and  PDF  divided  into  intervals. 


a  =  Pr(reject  Nol-f^o  true) 


(1.11) 


is  specified.  Observe  that 

a  =  Pr(x^  >  Mlffo)  =  (1  -  i^x^(M))  (1.12)  ’ 

where  denotes  the  distribution  function  of  the  statistic.  Since 

F^2(M)  =  l-a  (1.13) 

M  is  the  100  x  (1  —  a)  percentile  of  the  x^  statistic.  M  is  also  referred  to  as  the 
rejection  limit.  Values  of  M  are  tabulated  in  table  1.2  [12]  and  [13]. 

The  exact  distribution  of  the  statistic  x^  is  not  simple  and  depends  on  the  p,’s  in  the 
null  distribution  (Ho)  and  the  distribution  under  test.  However,  remarkably,  it  was 
found  that  these  difficulties  disappear  as  the  sample  size  n,  becomes  large  [12].  The 
distribution  of  x^  under  Ho  tor  large  n  is  approximately  one  of  the  family  of  chi-square 
distributions,  depending  on  the  number  of  intervals,  k,  but  not  on  the  distribution 
under  test.  This  family  of  distributions  is  characterized  by  the  number  of  degrees 
of  freedom,  defined  to  be  k-1.  Various  percentiles  of  the  chi-square  distribution,  for. 

selected  numbers  of  degrees  of  freedom,  are  given  in  table  1.2 
1.3.1  Example[l3] 

Two  hundred  measurements  of  viscosity  are  given  in  table  1.3.  The  table  gives 
frequencies  /,■  corresponding  to  the  ten  intervals.  Data  giv'en  in  the  table  can  be  used 
to  test  the  hypothesis  that  the  probability  density  function  (PDF)  from  which  they 
come  is  normal  with  mean  32  and  standard  deviation  1.8.  We  select  the  significance 
level  a  to  be  0.05,  so  that  M  equals  the  95‘^  percentile  of  the  x^  distribution.  Table 
1.3  shows  that  k,  the  number  of  intervals  is  10.  Consequently,  the  number  of  degrees 
of  freedom  given  by  A:  —  1  equals  9.  By  extracting  the  95^^  percentile  of  the  x^ 
distribution  with  9  degrees  of  freedom  from  table  1.2  it  is  seen  that  M  =  16.9. 

To  illustrate  the  computations  in  table  1.3,  the  entry  for  p4  is  obtained  from  the 
normal  distribution  function  by  means  of  eq.  1.6.  Hence, 
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Degrees 

of 

freedom 

P 

.01 

.025 

.05 

.10 

.70 

.80 

.90 

.95 

.975 

.99 

1 

.000 

.001 

.004 

.016 

1.0^ 

1.64 

2.71 

3.84 

5.02 

6.63 

-) 

.020 

.051 

.103 

.211 

2.41 

3.22 

4.61 

5.99 

7.38 

9.21 

.115 

.216 

.352 

.584 

3.66 

4.64 

6.25 

7.81 

9.35 

11.3 

4 

.297 

.484 

.711 

1.06 

4.88 

5.99 

7.78 

9.49 

11.1 

13.3 

5 

.554 

.S3 1 

1.15 

1.61 

6.06 

7.29 

9.24 

11.1 

12.8 

15.1 

6 

.S‘’2 

1.24 

1.64 

2.20 

7.23 

8.56 

10.6 

12.6 

14.4 

16.8 

7 

1.24 

1.69 

2.17 

2.83 

8.38 

9.80 

12.0 

14.1 

16.0 

18.5 

8 

1.65 

2.18 

2.73 

3.49 

9.52 

11.0 

13.4 

15.5 

17.5 

20.1 

9 

2.09 

2.70 

4.17 

10.7 

12.2 

14.:^ 

16.9 

19.0 

21.7 

10 

2.56 

3.25 

3.94 

4.87 

11.8 

13.4 

16.0 

18.3 

20.5 

23.2 

11 

3.05 

3.82 

4.57 

5.58 

12.9 

14.6 

17.3 

19.7 

21.9 

24.7 

12 

3.5" 

4.40 

5.23 

6.30 

14.0 

15.8 

18.5 

21.0 

23.3 

26.2 

13 

4.11 

5.01 

5.89 

7.04 

15.1 

17.0 

19.8 

22.4 

24.7 

27.7 

14 

4.66 

5.63 

6.57 

7.79 

16.2 

18.2 

21.1 

23.7 

26.1 

29.1 

15 

5.23 

6.26 

7.26 

8.55 

17.3 

19.3 

22.3 

25.0 

27.5 

30.6 

16 

5.81 

6.91 

7.96 

9.31 

18.4 

20.5 

23.5. 

26.3 

28.8 

32.0 

17 

6.41 

7.56 

8.67 

10.1 

19.5 

21.6 

24.8 

27.6 

30.2 

33.4 

18 

7.01 

8.23 

9.39 

10.9 

20.6 

22.8 

26.0 

28.9 

31.5 

34.8 

19 

7.63 

8.91 

10.1 

11.7 

21.7 

23.9 

27.2  . 

30.1 

32.9 

36.2 

20 

8.26 

9.59 

10.9 

12.4 

22.8 

25.0 

28.4 

31.4 

34.2 

37.6 

21 

8.90 

10.3 

11.6 

13.2 

23.9 

26.2 

29.6 

32.7 

35.5 

38.9 

22 

9.54 

11.0 

12.3 

14.0 

24.9 

27.3 

30.8 

33.9 

36.8 

40.3 

23 

10.2 

11.7 

13.1 

14.8 

26.0 

28.4 

32.0 

35.2 

38.1 

41.6 

24 

10.9 

12.4 

13.8 

15.7 

27.1 

29.6 

33.2 

36,4 

39.4 

43.0 

25 

11.5 

13.1 

14.6 

16.5 

28.2 

30.7 

34.4 

37.7 

40.6 

44.3 

26 

12.2 

13.8 

15.4 

17.3 

29,2 

31.8 

35.6 

38.9 

41.9 

45.6 

27 

12.9 

14.6 

16.2 

18.1 

30.3 

32.9 

36.7 

40.1 

43.2 

47.0 

28 

13.6 

15.3 

16.9 

18.9 

31.4 

34.0 

37.9 

41.3 

44.5 

48.3 

29 

14.3 

16.0 

17.7 

19.8 

32.5 

35.1 

39.1 

42.6 

45.7 

49.6 

30 

15.0 

16.8 

18.5 

20.6 

33.5 

36.2 

40.3 

43.8 

47.0 

50.9 

40 

22.1 

24.4 

26.5 

29.0 

44.2 

47.3 

51.8 

55.8 

59.3 

63.7 

50 

29.7 

32.3 

34.8 

37.7 

54.7 

58.2 

63.2 

67.5 

71.4 

76.2 

60 

37.5 

40.5 

43.2 

46.5 

65.2 

69.0 

74.4 

79.1 

83.3 

88.4 

Table  1.2:  Percentiles  of  the  Chi-Squared  Distribution 
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i 

interval 

Pi 

200p,' 

fi 

{fi  -  200pi)^  = 

Zi/{200pi) 

1 

<27.85 

0.0105 

2.10 

3 

0.81 

0.3857 

2 

27.85-28.95 

0.0346 

6.92 

7 

0.0064 

0.0009 

3 

28.95-30.05 

0.0943 

18.86 

25 

37.65 

1.1996 

4 

30.05-31.15 

0.1791 

35.82 

42 

38.10 

1.06 

5 

31.15-32.25 

0.2388 

47.76 

56 

67.90 

1.422 

6 

32.25-33.35 

0.2161 

43.22 

30 

174.50 

4.037 

~T1 

33.35-34.45 

0.1399 

27.98 

22 

35.80 

1.279 

8 

34.45-35.55 

0.0624 

12.48 

11 

2.19 

0.175 

9 

35.55-36.65 

0.0195 

3.90 

3 

0.81 

0.208 

10 

>36.65 

0.0048 

0.96 

1 

0.0016 

0.0017 

Table  1.3;  Measurements  of  viscosity  for  example  1.3.1 


Pr(30.05  <  .Y  <  31.15) 


^,31.15-32,  ^,30.05  -  32 

M — — )-^( 


1.8 


1.8 


) 


=  0.1791 


(1.14) 


where  Fo{x)  =  and 


$(x)  =  r  =  PriX  <  x).  (1.15) 

J-oo  \/2ir 

Values  of  $(a:)  are  obtained  from  a  table  for  the  standard  normal  distribution  function. 

The  statistic  is  obtained  by  summing  the  entries  in  the  last  column,  with  the 
result  =  10.16.  This  does  not  exceed  the  95'^*  percentile  of  the  chi-square  distribu¬ 
tion  with  9  “degrees  of  freedom”,  i.e.  The  chi-square  test,  therefore,  calls 

for  accepting  the  null  distribution  on  the  basis  of  the  given  data. 

Notice  that  the  chi-square  test  does  not  test  Fo(x)  but  only  the  p,  ’s.  In  particular, 
the  natural  order  of  the  intervals  does  not  enter  the  test.  Moreover,  Fo  is  not  the 
only  distribution  function  having  the  pi's  obtained  from  Fq.  Despite  these  minor 
objections,  the  chi-square  test  is  frequently  used  in  testing  a  continuous  distribution. 

The  chi-square  test  suffers  from  the  same  disadvantages  as  mentioned  earlier  for  the 
Kolmogorov-Smirnov  test.  In  addition,  it  has  one  more  disadvantage,  viz.,  it  requires 
a  large  sample  size  to  give  accurate  results. 
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1.4  Q-Q  (Quantile- Quantile)  Plot 

A  Q-Q  plot  is  a  special  plot  or  graphical  technique  which  can  be  performed  to 
assess  the  marginal  distribution  of  the  sample  observations.  Consider  a  set  of  data  of 
size  n  given  by  xi,  X2, . . . ,  x„.  Let  the  data  be  rank  ordered  such  that  X(i)  <  X(2)  < 
•  ■  •  <  X(n).  For  the  ordered  sample  X(j),  define 

P(i)  =  — - -  (1-16) 

where  the  1/2  is  introduced  as  a  “continuity  correction”  [14].  Let  Fx(x)  denote  the 
cumulative  distribution  function  of  the  data.  For  large  enough  sample  values  of  n,  it 
then  follows  that 

Fx{x(j))  =  Pr{X  <  X(j))  ft;  p(j).  (1.17) 

Denote  the  cdf  of  the  null  distribution  by  Fq{z).  The  quantile  of  Fo(z),  denoted  by 
^(j),  is  related  to  p(j)  by 

^o(9(j))  =  Pr{Z  <  q(j))  =  p(j).  (1.18) 

If  the  data  comes  from  the  same  distribution  as  the  null  distribution,  then 

^(j)  ~  ?(i) 

and  X(j)  can  be  interpreted  as  an  estimate  of  the  sample  quantile. 

A  Q-Q  plot  is  generated  using  the  following  steps: 

1)  Collect  n  data  points  xi,  X2,  - . . ,  x„. 

2)  Rank  order  the  data  such  that  X(i)  <  X(2)  <  • . .  <  X(„). 

3)  Define 

i  -  1/2  .  ,  . 

- ;  J  =  l,2,...,n. 

4)  Evaluate  the  quantile  defined  by 

^o(g(j))  =  P(i);  i  =  l,2,...,n. 
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j 

Ordered 

observations  X(j) 

Probability  levels 

P(i)  =  (J  -  l/2)/n 

Standard  normal 
quantiles  q(j) 

1 

-1.00 

0.05 

2 

0.15 

3 

0.16 

0.25 

0.116 

Kl 

0.41 

0.35 

0.396 

5 

0.62 

0.45 

0.649 

6 

0.80 

0.55 

0.891 

B 

1.26 

0.65 

1.144 

8 

1.54 

0.75 

1.424 

9 

1.71 

0.85 

1.755 

10 

2.30 

0.95 

2.366 

Table  1.4:  Observation  Table  for  Example  1.4.1 


5)  Plot  the  pair  of  points 

J  —  1?  2, . . . ,  n. 

When  the  data  comes  from  the  null  distribution,  the  Q-Q  plot  is  likely  to  approxi¬ 
mate  a  straight  line  through  the  origin  at  45°. 

1.4.1  Example[l4] 

A  sample  of  n  =  10  observations  gives  the  values  tabulated  in  the  2"^^  column  of  the 
table  1.4.  The  sample  mean  and  the  sample  variance  are  m  =  0.77  and  =  0.9414 
respectively.  The  values  of  p(j)  are  computed  in  the  3’’'^  column.  Finally,  taking  the 
normal  distribution  with  mean  m  and  variance  as  the  null  distribution,  the  corre¬ 
sponding  quantiles  q^j)  are  evaluated  in  the  4'^  column.  For  example,  corresponding 
to  p(9)  =  0.85 

rl.775  I  (z-0.771^ 

Fo(9(9))  =  Pr{Z  <  1.775)  =  /  , .  e  1  6828  dz  =  0.85.  (1.20) 

J-<x>  ■\/2Trcr'^ 

Consequently,  =  1.775. 

The  Q-Q  plot  for  the  above  data,  which  is  a  plot  of  the  ordered  data  against 
the  normal  quantiles  q^jj,  is  shown  in  fig.  1.4.  The  pair  of  points  a;(j))  lie  very 
nearly  along  a  straight  line  at  45°  and  we  accept  the  notion  that  these  are  normally 
distributed  with  mean  =  0.77  and  variance  =  0.9414. 

The  straightness  of  the  Q-Q  plot  can  be  evaluated  by  calculating  the  correlation 
coefficient  of  the  points  in  the  plot.  The  correlation  coefficient  for  the  Q-Q  plot  is 
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2 


1.5 


Normal  ^ 
Quantiles 

0.5 


0 


-0.5 


-1 

-1  -0.5  0  0.5  1  1.5  2  2.5 

Ordered  data  X(j) 


n - 

1 - 

1 

— 
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o 

0 

- 
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o 

o 

- 

o 

o 
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O 

j _ 

- 1 _ L 
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- 

Figure  1.4;  The  Q-Q  Plot  for  Example  1.4.1. 
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Sample  size 
n 

Signifj 

0.01 

cance  le 

0.05 

vels  a 
0.10 

5 

0.8299 

0.8788 

0.9032 

10 

0.8801 

0.9198 

0.9351 

15 

0.9126 

0.9389 

0.9503 

20 

0.9269 

0.9508 

0.9604 

25 

0.9410 

0.9591 

0.9665 

30 

0.9479 

0.9652 

0.9715 

35 

0.9538 

0.9682 

0.9740 

40 

0.9599 

0.9726 

0.9771 

45 

0.9632 

0.9749 

0.9792 

50 

0.9671 

0.9768 

0.9809 

55 

0.9695 

0.9787 

0.9822 

60 

0.9720 

0.9801 

0.9836 

75 

0.9771 

0.9838 

0.9866 

100 

0.9822 

0.9873 

0.9895 

150 

0.9879 

0.9913 

0.9928 

200 

0.9905 

0.9931 

0.9942 

300 

0.9935 

0.9953 

0.9960 

Table  1.5:  Critical  Points  for  the  Q-Q  Plot  Correlation  Coefficient  Test  for  Normality 


approximated  by 

.  Ej=i(^-(j)  -  ^)(g(j)  -  <?) 

-  A)VS”=.(9U)  -  9)" 

where  q  is  the  sample  mean  of  the  quantiles  q^^y,  j  =  1,2,. . .  ,n.  Formally,  we  select 
the  null  hypothesis  at  a  significance  level  a  il  rq  exceeds  a  critical  value  denoted  by  M 
[14].  The  values  of  M  have  been  evaluated  for  the  normal  distribution  and  tabulated 
in  table  1.5  [14]  for  different  sample  sizes  and  significance  levels. 

For  the  above  example  we  select  a  =  0.10.  Also,  using  the  information  from  table 
1.4,  we  find  that  the  mean  of  the  sample  quantiles  and  standard  normal  quantiles 
are,  respectively,  m  =  0.77  and  ^  =  0.  Using  eq.(1.21),  we  find  that  the  correlation 
coefficient,  fg,  is  found  to  be  0.9943.  Referring  to  table  1.5,  we  find  that  corresponding 
to  n  =  10  and  a  —  0.10,  the  critical  point  M,  for  the  Q-Q  plot  correlation  coefficient 
test  for  normality  is  0.9351.  Since  fq  >  0.9351,  we  accept  the  hypothesis  of  normality. 

1.5  P-P  (Probability-Probability)  Plot 

The  P-P  plot  is  another  graphical  technique  which  is  performed  for  random  data 
analysis.  Just  as  with  the  Q-Q  plot  we  consider  a  set  of  data  of  size  n  given  by 
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Xi,  X2, . . . ,  Xji-  The  data  is  rank  ordered  such  that  <  x^2)  ^  <  x^^n)-  Proceeding 

as  we  did  with  the  Q-Q  plot,  define  for  the  ordered  sample,  x^j), 

P{i)  =  (1-22) 

where  the  1/2  is  introduced  as  a  “continuity  correction”  [14].  Let  Ta-(x)  denote  the 
cumulative  distribution  function  of  the  data.  From  the  Q-Q  plot  we  know  that  the 
X(j)’s  are  the  sample  quantiles.  Denote  the  cdf  of  the  null  distribution  by  Fo(z).  Then 
is  defined  to  be  the  probability  such  that 

Fo(a;(j))  =  Pr{Z  <  X(j))  =  (1.23) 

If  the  data  comes  from  the  same  distribution  as  the  null  distribution,  it  is  likely  that 


Pxo,  P(j) 

and  p(j)  can  be  interpreted  as  an  estimate  of  the  probability 
A  P-P  plot  is  generated  using  the  following  steps: 

1)  Collect  n  data  points  a;i,X2, •  •  •  ,a:„. 

2)  Rank  order  the  data  such  that  <  X(^2)  <  •  •  •  ^  ^’(n)- 

3)  Define 


(1.24) 


j  -  1/2  . 

P{j)  =  — - - ;  j  =  l,2,...,n. 


4)  Evaluate  the  probability  defined  by 


^o(a:(j))  =Px(_,);  ;  =  1,2,...,; 


5)  Plot  the  pair  of  points 


(P®o)’PU))i  i  =  l,2,...,n. 


When  the  data  comes  from  the  null  distribution,  the  P-P  plot  is  likely  to  approxi¬ 
mate  a  straight  line  through  the  origin  at  45°. 
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1.5.1  Example 

We  take  the  example  used  with  the  Q-Q  plot  and  find  the  P-P  plot  of  the  given 
data.  The  observations  are  tabulated  in  the  2""^  column  of  table  1.6.  Values  of  p(j)  are 
computed  in  the  2”*^  column.  The  sample  mean  and  the  sample  variance  are  m  =  0.77 
and  =  0.9414.  Finally,  taking  the  normal  distribution  with  mean  m  and  variance 
(7^  as  the  null  distribution,  the  corresponding  probabilities  are  evaluated  in  the 
column.  For  example,  corresponding  to  X(7^  =  1.26 

/•1.26  I  (i-0,77i2 

Fo{xi7))  =  Pr{Z  <  1.26)  =  /  -==t—T^dz  =  0.6932.  (1.25) 

7-00  v27r(7^ 

Consequently,  =  0.6932 

The  P-P  plot  for  the  above  data,  which  is  a  plot  of  the  v'^alues  p(j)  against  the 
normal  probabilities  Prj^),  is  shown  in  fig.  1.5.  The  pair  of  points  lie  very 

nearly  along  a  straight  line  at  45°  and  we  accept  the  notion  that  these  are  normally 
distributed  with  mean  =  0.77  and  variance  =  0.9414. 

The  straightness  of  the  P-P  plot  can  be  evaluated  by  approximating  the  correlation 
coefficient 


f  —  P){P^u)  26) 

where  p  and  px  are  the  sample  means  of  p(j)  and  p^^^^ ,  respectively,  with  j  =  1, 2, . . . ,  n. 
Unfortunately,  a  table  for  the  critical  value  M  for  different  values  of  the  significance 
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P-P  Plot  for  Example  1.5.1. 


level,  a,  was  not  found  in  the  literature.  Nevertheless,  if  fp  is  close  to  unity,  and 

Px^j^  are  highly  correlated  and  although  a  significance  level  cannot  be  specified,  it  is 
likely  that  the  data  can  be  approximated  with  the  null  distribution.  For  this  example, 

rp  =  0.9960.  (1.27) 

Since  rp  «  1,  it  is  concluded  that  the  data  is  statistically  consistent  with  the  normal 
distribution  having  mean  m  —  0.77  and  variance  cr^  =  0.9414 

An  attractive  property  of  the  Q-Q  plot  is  that  it  is  invariant  to  a  linear  transforma¬ 
tion.  Specifically,  the  Q-Q  plot  of  a  linear  function  of  X(j)  is  again  a  straight  line  at 
45°.  However,  this  time  the  line  need  not  pass  through  the  origin.  P-P  plots  do  not 
have  this  property.  The  main  drawback  of  these  plots  is  their  weak  performance  for 
small  and  moderate  sample  sizes.  Also,  generalization  of  the  Q-Q  plot  to  multivariate 
distributions  is  not  straightforward.  On  the  other  hand  P-P  plots  can  be  applied  to 
the  multivariate  situation.  Although  a  statistic  exists  for  evaluating  the  straightness 
of  the  Q-Q  plot  when  the  null  distribution  is  standard  normal,  this  statistic  is  not 
readily  available  for  other  distributions.  Consequently,  the  Q-Q  plots  and  the  P-P 
plots  do  not  readily  offer  a  quantative  Goodness  of  Fit  test  and  the  decision  is  mostly 
made  on  a  subjective  basis. 
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Chapter  2 

The  Ozturk  Algorithm 


2.1  Introduction 

In  testing  a  null  hypothesis  for  a  distributional  assumption  against  an  unspecified 
alternative  there  is  generally  no  uniformly  most  powerful  or  optimal  test  [2].  Due  to 
this,  various  test  procedures  have  been  developed  for  assessing  these  distributional 
assumptions.  Under  certain  conditions,  (i.e.  for  a  specified  null  hypothesis,  a  speci¬ 
fied  sample  size,  and  a  pre-determined  level  of  significance)  one  test  procedure  may 
be  shown  to  be  more  powerful  than  the  other  existing  procedures.  Besides  the  power 
consideration  of  a  given  test,  computational  simplicity,  desirable  distributional  prop¬ 
erties  of  the  test  statistic  and  the  generality  of  the  test  procedure  are  some  of  the 
important  properties  to  be  considered. 

Chapter  1  gave  a  brief  overview  of  some  of  these  tests.  The  f^st  has  been 
widely  used  for  assesing  the  distributional  assumptions  because  of  its  generality  and 
its  computational  simplicity  [2].  However,  the  choice  of  class  intervals  for  computing 
the  test  statistic  is  arbitrary  and  the  procedure  can  be  used  only  for  large  sample  sizes. 
Q-Q  plots  and  P-P  plots  are  among  the  most  widely  used  graphical  procedures  for 
making  asessment  about  the  random  data.  But  their  performance  is  weak  for  small 
and  moderate  sample  sizes.  Also,  generalizations  of  Q-Q  plots  to  the  multivariate- 
distributions  are  not  simple  [5],  [6].  As  described  in  [9],  [12]  and  [13],  the  Kolmogorov- 
Smrinov  test,  which  is  based  on  the  empirical  distribution  function  of  the  sample  and 
the  null  distribution,  is  widely  used  too.  In  fact,  comparative  studies  have  shown 
that  the  Kolmogorov-Smirnov  statistic  has  higher  power  than  the  statistic  for 
many  alternatives  [2].  There  are  many  other  tests  such  as  the  W  test  (by  Shapiro 
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and  Wilk),  Anderson’s  A  test  [2],  etc. 

All  these  tests  are  Goodness  of  Fit  tests. To  within  a  certain  confidence  level,  these 
tests  provide  information  about  whether  a  set  of  random  data  is  statistically  consistent 
with  a  specified  null  distribution.  However,  if  the  specified  distribution  is  rejected, 
these  tests  give  no  clue  about  the  alternative  underlying  distribution  of  the  data. 
Thus,  we  need  to  have  a  priori  knowledge  about  the  random  data  to  be  able  to  use 
these  tests.  In  practice,  a  lot  of  times  we  have  no  a  priori  knowledge  about  the  random 
signals.  For  example,  the  clutter  PDF  encountered  in  radar  signal  processing  is  not 
known  a  priori.  Moreover,  a  lot  of  these  tests  require  a  large  number  of  observations  to 
give  accurate  results.  To  get  these  many  observations  may  prove  costly  in  a  real  world 
situation.  Consequently,  a  scheme  is  necessary,  that  not  only  performs  the  Goodness 
of  Fit  test  but  also  approximates  the  PDF  for  small  number  of  observations. 

A  new  algorithm  based  on  sample  order  statistics  has  been  developed  in  [1]  through 
[3]  and  has  been  reported  in  [10]  for  univariate  distribution  approximation.  This 
algorithm  has  two  modes  of  operation.  In  the  first  mode,  the  algorithm  performs 
a  Goodness  of  Fit  test.  Specifically,  the  test  determines,  to  a  desired  confidence 
level,  whether  the  random  data  is  statistically  consistent  with  a  specified  probability 
distribution.  In  the  second  mode  of  operation,  the  algorithm  approximates  the  PDF  • 
underlying  the  random  data.  In  particular,  by  analyzing  the  random  data  and  without 
any  a  priori  knowledge,  the  algorithm  identifies  from  a  stored  library  of  PDFs  that 
density  function  which  best  approximates  the  data.  Estimates  of  the  location,  scale, 
and  shape  parameters  of  the  PDF  are  provided  by  the  algorithm.  The  algorithm  is 
typically  found  to  work  well  for  observation  sizes  of  the  order  of  75-100. 

In  this  chapter  we  present  the  Ozturk  algorithm.  It  will  be  demonstrated  through 
examples  that  the  algorithm  can  be  used  to  test  for  any  distributional  assumption 
(not  limited  to  location-scale  family)  including  univariate  and  multivariate  random 
variables. 

2.2  Definitions 

Let  F'r(y)  denote  the  PDF  of  a  random  variable  Y.  Consider  the  linear  transfor¬ 
mation  defined  by 


x  =  I3y  + a. 


(2.1) 
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The  PDF  of  X  is  given  by 


fxM  =  (2.2) 

where  a  and  {3  are  defined  to  be  the  location  and  scale  parameters  of  fx{x),  respec¬ 
tively.  The  mean  and  the  variance  of  the  random  variable  x  are  given  by 


III  =  £|xl 

<tI  =  E[{x  -  (2.3) 

where  E  is  the  expectation  operator. 

Although  the  mean  and  the  variance  are  related  to  the  location  and  scale  parame¬ 
ters,  note  that  the  location  parameter  is  not  the  mean  value  and  the  scale  parameter 
is  not  the  square  root  of  the  variance,  in  general.  However,  for  a  standard  Gaussian 
PDF  friy),  for  which  the  mean  is  zero  and  variance  unity,  the  location  parameter  is 
the  mean  of  X  and  the  scale  parameter  is  the  standard  deviation  (square  root  of  the 
variance)  of  X. 

The  coefficient  of  skewness  Q3,  and  the  coefficient  of  kurtosis  04,  of  X  are  defined 
to  be 


03 


0:4 


E\{x-ixx)^] 

<’1 


(2.4) 


It  is  readily  shown  that  03  and  04  are  invariant  to  the  values  of  pix  and  (Tx-  For 
any  PDF  that  is  symmetric  about  the  mean,  03  =  0.  For  the  case  of  the  Gaussian 
distribution,  a3  =  0  and  —  3. 


2.3  The  Ozturk  Algorithm 

Any  distribution,  or  a  family  of  distributions,  can  be  represented  by  a  single  point  or 
by  a  region  on  an  03  —  04  plane,  respectively,  where  is  the  coefficient  of  skewness 
and  a4  is  the  coefficient  of  kurtosis  (see  for  example  [15],  p.l4).  A  set  of  random 
data  can  also  be  represented  by  a  point  whose  co-ordinates  are  given  by  the  sample 
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estimates  of  Q3  and  04.  Then  the  best  candidate  for  the  underlying  true  distribution 
can  be  identified  tobe  the  nearest  neighbour  distribution  on  the  chart.  Although  such 
a  chart,  based  on  the  coefficient  of  skewness  and  kurtosis,  provides  a  useful  way  of 
characterizing  the  distributions,  its  use  is  limited  by  the  fact  that  the  moments  of 
some  distributions  do  not  exist.  Other  drawbacks  of  this  approach  are 

1.  Estimates  of  03  and  Q4  are  highly  sensitive  to  extreme  observa¬ 
tions. 

2.  Estimates  of  these  moments  are  highly  biased  for  small  sample 
sizes  [4]. 

3.  The  moment  estimators  are  greatly  affected  by  outliers. 

In  this  chapter  we  introduce  the  Ozturk  Algorithm,  a  general  graphical  technique 
which  works  in  two  specific  modes. 

1.  In  the  first  mode  it  performs  a  formal  Goodness  of  Fit  test  for  a 
specified  null  distribution. 

2.  In  its  second  mode  it  provides  a  graphical  representation  that  gives 
insight  into  what  distribution  best  approximates  the  data  set  and 
thus  provides  a  way  of  characterizing  the  data. 

2.3.1  Goodness  of  Fit  Test 

The  Goodness  of  Fit  test  is  a  complex  algorithm  which  determines  whether  or  not 
the  set  of  data  samples  provided  to  the  algorithm  is  statistically  consistent  with  a 
specified  distribution  (the  null  hypothesis).  Using  the  standard  normal  distribution 
with  zero  mean  and  unit  variance  as  the  reference  distribution,  the  standardized  sam¬ 
ple  order  statistics  are  represented  by  a  system  of  linked  vectors.  The  terminal  point 
of  the  linked  vectors,  as  well  as  the  shape  of  their  trajectories,  are  used  in  determining 
whether  or  not  to  accept  the  null  hypothesis.  In  its  present  form  the  algorithm  uses 
the  standard  Gaussian  distribution  as  the  reference  distribution.  However,  any  other 
distribution  could  be  used  as  the  reference  distribution.  The  null  hypothesis  is  the 
distribution  against  which  the  sample  data  is  to  be  tested.  Note  that  the  reference 
distribution  need  not  be  the  same  as  the  null  distribution. 
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We  begin  by  introducing  several  sample  order  statistics  used  in  the  algorithm  and 
then  proceed  to  explain  the  Goodness  of  Fit  test  procedure 
Consider  the  following  three  sets  of  data  size  n: 

1.  A  sample  data  set 

with  mean  and  standard  deviation  given  by  fix  and  cTx- 

2.  A  null  hypothesis  data  set 

Zli  Z2,  ^3, . . .  ^Zn 

generated  from  any  available  distribution  against  which  the  sam¬ 
ple  set  will  be  tested.  The  mean  and  standard  deviation  of  this 
data  set  are  defined  to  be  and  <7-,  respectively. 

3.  A  reference  distribution  data  set 


Wi,W2,W3,...,W, 

generated  from  the  standardized  Gaussian. 

Let  Xi-n  <  ^2:71  <  •  •  •  <  Xn-.n  denote  the  ordered  set  of  samples  obtained  by 
ordering  Xi',i  =  1,2,  where  Xi-.n  is  the  smallest  data  sample.  Similiarly,  the 

other  two  data  sets  are  ordered  resulting  in  the  three  ordered  data  sets 


Define 


AT:n. 

)  X2:nj  X^int  - 

X 

•  *  1 

Zlm 

1  Z2-.n1  Zs-.ni  • 

•  •  1  Zn-.n 

VF2:„,W3:„,. 

w 

•  •  1  ••  n:n 

:n 

- - -  J  1 

11 

(2.5) 


(2.6) 
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where  =  ^Xijn  is  the  sample  mean  and  —  Hx)^l{n  —  is  the 

sample  standard  deviation.  These  are  the  standardized  order  statistics  of  the  sample 
data.  For  the  null  hypothesis,  a  Monte  Carlo  simulation  consisting  of  2,000  trials  is 
utilized.  The  estimate  of  the  expected  value  of  the  standardized  order  statistic  is ' 
defined  as 


=  - ’  ^  =  l,2,...,n.  (2.7) 

where  (Z,;„)fc  denotes  the  order  statistic  from  the  Monte  Carlo  trial,  and  flz  and 
dz  denote  the  sample  mean  and  sample  standard  deviation.  Also,  m,;„  is  defined  as  the 
estimate  of  the  expected  value  of  the  order  statistic  of  the  reference  distribution, 
the  standardized  Gaussian.  Using  2,000  Monte  Carlo  trials, 


2000 

=  :^  ^(lU'.:n)A:;  i=l,2,...,n  (2.8) 

where  (lT,;„);t  denotes  the  order  statistic  from  the  k'^'^  Monte  Carlo  trial  of  the 
reference  distribution.  When  the  null  hypothesis  is  the  reference  distribution,  the 
standardized  Gaussian,  then 


fi,n  «  m,:„.  (2.9) 

The  Goodness  of  Fit  test  proceeds  by  joining  together  two  sets  of  n  linked  vectors, 
one  for  the  sample  data  and  one  for  the  null  hypothesis.  The  linked  vector  in  each 
set  is  characterized  by  its  length  and  orientation  with  respect  to  the  horizontal  axis. 
For  the  sample  data,  the  length  of  the  vector,  a,-,  is  obtained  from  the  magnitude 
of  the  standardized  order  statistic  of  the  data,  while  its  angle  or  orientation,  6,,  is 
related  to  m,:n-  More  specifically,  for  the  sample  data 


Oi  =  ;  <f>{x)  =  -^=  [  exp{-\-)dt.  (2.10) 

V  27r  J—oo  2 


(t>x  is  the  cumulative  distribution  of  the  standard  Gaussian  distribution.  We  define 
the  sample  points  Qk  in  a  two  dimensional  plane  (C/,  V)  by 
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Qk  =  iUk,Vk)  ; 


(2.11) 


where  Uq  =  Vq  =  0  and 

1^-1008(00 

^  »=1 

=  7E!>^-Isin(0.)  (2.12) 

i=i 

k  =  1, 2, . . . ,  n. 

Similiarly,  for  the  null  hypothesis  the  length  of  the  vector,  fc;,  is  obtained  from 
the  magnitude  of  the  standardized  order  statistic  of  the  null  data  set.  Specifically, 
for  the  null  data 

1).  =  0.  -  TT(p{mi,n).  (2.13) 

n 

Using  the  same  two  dimensional  plane,  we  plot  the  sample  points  for  the  null  distri¬ 
bution  defined  by 

Qok  =  (Uofc,  Vofc)  ;  A:  =  1, 2, . . . ,  n  (2-14) 

where,  Uqq  =  Vqo  =  0  and 

1  ^  - 

Uok  =  TY.\Ti:n\cOs{9i) 

*  ,=1 

1  - 

l^^  =  7El^‘-|sin(0O  (2.15) 

^  i=i 

k  =  1, 2, . . . ,  n. 

Note  that  the  angle  9  remains  the  same  for  both  sets  of  linked  vectors.  However, 
the  magnitude  of  the  linked  vector  for  the  sample  data  is  a,-  whereas  it  is  6;  for  the 
null  distribution.  The  angle  9i  is  solely  dependent  on  the  reference  distribution  while 
the  magnitudes  |yi;„|  and  |r,:„|  are  solely  dependent  on  the  sample  data  and  null  data 
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sets,  respectively.  In  particular,  for  the  sample  linked  vector,  a,  is  dependent  on  the 
standardized  order  statistic  of  the  sample  data  set  whereas  for  the  linked  vector 
of  the  null  hypothesis,  6,  is  dependent  on  the  estimate  of  the  expected  value  of  the 
standardized  order  statistic  of  the  Monte  Carlo  simulation  of  the  null  distribution. 

Although  Yi:n  and  r,;„  are  ordered  statistics  from  the  smallest  to  the  largest,  note 
that  the  magnitudes  of  Yi^n  and  Ti^n  are  not.  In  fact,  with  increasing  i,  and  iTjinl 
would  begin  large,  decrease  to  approximately  zero  and  then  increase  again. 

The  sample  and  null  linked  vectors,  respectively,  are  drawn  by  joining  the  points 
(QijQi-i)  (<5oti  <5o(i-i))-  It  should  be  noted  that  the  and  Qon  given  in  equa¬ 
tions  (2.11)  and  (2.14)  represent  the  terminal  point,  respectively,  of  the  linked  vectors 
defined  above.  Fig.  2.1  shows  the  two  sets  of  linked  vectors  obtained  when  both  the 
sample  and  null  data  sets  are  obtained  from  the  the  Gaussian  distribution  with  n  =  6 
and  n  =  50.  The  solid  curves  in  fig. 2.1  show  the  linked  vector  for  the  sample  distri¬ 
bution  while  the  dashed  curves  show  the  ideal  linked  vector  for  the  null  distribution. 
When  the  length  n,  of  the  data  set,  is  large  (on  the  order  of  50  points),  then  the 
linked  vector  is  a  smooth  arc,  as  seen  in  fig.2.1. 

For  a  typical  set  of  ordered  data  samples  drawn  from  the  null  distribution,  it  is 
reasonable  to  expect  that  the  sample  linked  vectors  would  follow  the  null  linked 
vectors  closely.  If  the  ordered  set  of  samples  is  not  from  the  null  distribution,  then 
the  sample  linked  vectors  are  not  expected  to  follow  the  path  of  the  null  linked  vectors 
closely.  Hence,  the  procedure  provides  visual  information  about  how  well  the  ordered 
set  of  data  fit  the  null  distribution.  However  this  is  not  an  ad  hoc  statistical  procedure. 
As  shall  be  seen  later  on,  we  do  construct  test  statistics  to  present  a  formal  way  of 
performing  the  Goodness  of  Fit  test  to  determine  whether  the  data  set  is  statistically 

consistent  with  the  null  hypothesis. 

2.3. 1.1  Properties  of  the  test  statistic  Qon 

An  important  property  of  the  Qon  statistic  is  that  it  is  invariant  under  linear 
transformation.  In  particular,  we  consider  the  standardization  used  in  eq.(2.6).  Let 
Si  =  cXi  -f  d  where  c  and  d  are  constants.  Let  fig  and  cr^  denote  the  mean  and 
standard  deviation  of  the  samples.  Si,  respectively.  Then,  it  is  readily  shown  that 
I  I  “  I  !•  invariance  property  follows  as  a  consequence.  The  advantage  of 
this  property  is  that  the  PDF  of  Qon  =  {Uoni  Fon)  for  a  given  sample  set  and  reference 
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distribution  depends  only  on  the  sample  size  n  and  is  unaffected  by  the  location  and 
scale  parameters.  The  distributional  properties  of  this  statistic  for  testing  normality 
is  studied  by  Ozturk  and  Dudewicz  in  [3]. 

The  exact  sampling  distribution  of  Qon  is  usually  difficult  to  obtain.  However, 
the  empirical  distribution  of  the  test  statistic  Qon  was  obtained  via  Monte  Carlo 
experimentation  by  Ozturk  and  Dudewicz  in  [3]  .  Using  the  means,  variances  and 
coefficients  of  skewness  and  kurtosis  of  f7o„  and  Uon  based  on  50,000  samples  for  values 
of  n  from  3  till  100,  they  found  that  the  distributions  of  Uon  and  Von  approach  the 
normal  distribution  even  for  moderate  sample  sizes.  The  distributional  properties  of 
the  statistic  Qon  =  (Uon,  Vo„)  for  testing  normality  is  studied  by  Ozturk  and  Dudewicz 
in  [3].  Some  of  the  empirical  results  obtained  by  them  for  the  statistic  Qon  for  standard 
normal  distribution  for  3  <  <  100  are  given  below. 


E{Uon) 

EiVon) 

E{UonVon) 

VariUon) 

yar[VQn) 


0 


IJ-V 


^  0.326601  + 


0.412921 

n 


0 


0.02123  0.01765 

- 1 - 2 - 

n 

0.04427  _  0.0951 
n 


(2.16) 


(2.17) 


Also,  it  was  found  empirically,  for  n  >  10.  that,  f/on  and  Von  are  approximately 
bivariate  normal. 

An  interesting  property  of  this  algorithm  is  that  any  one  of  the  points  Qok’,  k  - 

1, 2, . . . ,  n,  or  a  selected  group  of  these  points  can  be  used  as  a  test  statistic  to  establish 

a  formal  test.  The  algorithm  in  its  present  form  proposes  the  general  statistic  Qon  as 

the  test  statistic  for  testing  the  null  hypothesis. 

2. 3. 1.2  Basic  Concept  of  the  Confidence  Contours 

The  algorithm  provides  quantitative  information  as  to  how  consistent  the  sample 

data  set  is  with  the  null  hypothesis  distribution  by  the  use  of  the  confidence  contours. 

An  example  of  these  contours  is  shown  in  fig.2.1.  If  the  end  point  of  the  sample  data 

linked  vector  curve  falls  within  one  or  more  of  these  contours,  then  the  sample  data 

set  is  said  to  be  statistically  consistent  with  the  null  hypothesis  at  a  confidence  level 


31 


based  on  the  confidence  contours.  If  the  sample  data  set  is  truly  consistent  with  the 
null  hypothesis,  note  that  the  sample  linked  vector  is  likely  to  closely  follow  the  null 
linked  vector. 

Now  consider  the  linked  vector  for  the  null  hypothesis  which  is  based  on  the  stan¬ 
dardized  expected  values  of  the  order  statistic,  Z,  for  2,000  Monte  Carlo  simulations. 
The  test  statistic  Qom  found  by  computing  the  expected  value  of  2,000  end  points 
of  the  2,000  linked  vectors  provided  by  the  Monte  Carlo  simulation,  is  random.  The 
coordinates  of  Qon,  Uon  and  Vo„,  may  or  may  not  be  bivariate  Gaussian. 

When  Uon  and  Vo„  are  bivariate  Gaussian,  the  confidence  contours  of  the  null  hy¬ 
pothesis  are  readily  determined.  A  three  dimensional  bell  shaped  bivariate  Gaussian 
curve  is  fitted  to  the  2,000  end  points  arising  from  the  Monte  Carlo  simulation.  The 
elliptical  contours  of  this  distribution  are  plotted  for  various  parameters  of  the  signif¬ 
icance  level  a  (eg.  0.01,  0.05,  0.1)  where  a  is  defined  as  the  conditional  probability 
that  Qon  falls  outside  the  specified  ellipse  given  that  the  data  comes  from  the  null 
distribution.  (1  —  a)  is  called  the  confidence  level  and  the  corresponding  contour  is 
called  the  100  x  (1  —  a)  percent  confidence  contour.  Note  that  (1  —  a)  is  the  condi¬ 
tional  probability  tha  Qon  falls  inside  the  specified  ellipse  given  that  the  data  comes 

» 

from  the  null  distribution. 

This  could  be  done  for  any  of  the  n  points  of  the  ordered  statistic,  Z,  along  the 
null  linked  vector.  Thus,  more  than  one  set  of  confidence  contours  could  be  created 
if  there  are  more  than  one  test  statistic.  Then,  if  the  sample  data  is  truly  consistent 
with  the  null  hypothesis,  the  sample  data  linked  vector  is  likely  to  pass  through  a 
series  of  confidence  contours  determined  from  the  distributions  of  the  test  statistics. 
However,  it  was  found  to  be  unnecessary  to  clutter  up  the  graphics  with  so  m.any 
contours,  as  the  human  eye  can  readily  detect  whether  or  not  the  linked  vectors  are 
closely  following  the  same  trajectory.  The  option  of  using  more  than  one  test  statistic 
is  provided  in  the  algorithm. 

Note  that  the  average  value  of  the  test  statistic,  Qon,  of  the  null  distribution  is  at  the 
center  of  the  contours.  Thus,  the  closer  the  end  point  of  the  sample  data  linked  vector 
is  to  the  center  of  the  confidence  contour,  the  more  likely  it  is  that  the  sample  data  is 
coming  from  the  null  hypothesis.  As  the  significance  level  decreases,  the  confidence 
level  increases  and  the  probabillity  that  Qon  will  fall  within  the  corresponding  ellipse 


32 


will  also  increase.  This  results  in  the  fact  that  the  size  of  the  confidence  contours 
increase  as  the  confidence  level  increases. 

For  a  given  sample  size,  n,  the  angle  of  any  linked  vector  depends  solely  on 
the  reference  distribution  which  remains  unchanged  throughout.  Consequently,  for  a 
given  value  of  sample  size,  n,  and  for  a  given  null  hypothesis,  values  for  the  magnitude 
and  angle  of  the  points  {UQk,Vok)  on  the  null  linked  vector,  k  =  1,2,. . .  ,n,  may  be 
tabulated.  This  table,  which  is  dependent  on  n  and  the  null  hypothesis,  could  be 
stored  and  recalled  when  desired.  This  can  significantly  reduce  the  computational 
requirements. 

2. 3. 1.3  Determining  Confidence  Contours 

As  described  earlier,  the  confidence  contours  are  contours  from  the  bivariate  proba¬ 
bility  density  function  of  the  end  point  coordinates  used  to  determine  the  test  statistic 
Qon-  These  2,000  end  points  are  obtained  from  the  Monte  Carlo  simulation.  Plotting 
confidence  contours  is  usually  not  easy  when  the  joint  distribution  is  not  bivariate 
normal.  Further,  in  order  to  analytically  determine  the  confidence  contours,  the  joint 
PDF  of  Uon  and  Von  must  be  known  [4].  However,  it  is  difficult  to  analytically  deter¬ 
mine  this  joint  PDF.  Consequently,  a  normality  transformation  is  made  on  the  end 
point  coordinates  Uon  and  Fon  to  obtain  statistics  rou  =  tpiiUon)  and  rov  =  '<p2iVon) 
where  t^i(.)  and  are  functions  operating  on  Uon  and  1^„,  respectively.  A  family 
of  distributions  called  the  Johnson  System  is  used  to  perform  the  transformation  on 
Uon  and  Von  so  as  to  obtain  a  bivariate  normal  distribution. 

The  Johnson  system  of  distributions  is  a  flexible  family  of  distributions  having 
four  parameters.  This  system  is  used  to  summarize  a  set  of  data  by  means  of  a 
mathematical  function  which  will  fit  the  data.  The  system  proposed  by  Johnson 
contains  three  families  of  distributions  which  are  obtained  by  transformations  of  the 
form 


R  =  j  +  rjfi{G;X,e);i  =  h2,3  (2.18) 

where  is  a  standard  normal  variable  and  G  is  the  random  variable  on  which  the 
transformation  is  performed.  7,77,  A,  and  e  are  four  parameters  of  the  Johnson  system 
of  distributions.  In  particular,  let 
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(2.19) 


-Rou  =  7i +»?i/i(C^n;  Ai,ei) 

Rov  —  72  +  V'ifiiVoni  A2,  62) 
i  =  1,2,3 

where  /,•  ;  i  =  1,2, 3  represent  the  following  three  functions  suggested  by  Johnson: 

fi{g;X,€)  -  (2.20) 

denotes  the  Su  distribution, 

/2(^;A,e)  =  ln(--^ — ^),  €<^<e  +  A  (2.21) 

X+t-g  ^  ^ 

denotes  the  Sb  distribution,  and 

fsig-,  A,  e)  =  ln(^^),  g  >  e  (2.22) 

denotes  the  Sl  distribution. 

Note  that  fi{g;X,e),i  -  1,2,..., 3,  are  single-  valued  monotonically  increasing 
functions  for  the  allowed  ranges  of  g.  Sl  is,  in  essence,  a  three  parameter  distribution 
since  the  parameter  A  can  be  eliminated  by  letting  7*  =  7  —  7  In  A  so  that  r  = 
7  +7/  ln(^  —  t).  Sb  is  a  distribution  bounded  on  (e,  e-f  A)  and  the  Su  is  an  unbounded 
distribution.  In  a  plot  of  the  third  and  fourth  order  standardized  moments  where 
is  plotted  versus  0:4,  the  chosen  functions  are  such  that  the  Sl  distributions  form  a 
curve  dividing  the  (y^oa,  a4)  plane  in  two  regions.  The  Sb  distributions  lie  in  one  of 
the  regions  and  the  Su  lie  in  the  other. 

In  using  this  system  of  transformations,  the  first  step  is  to  determine  which  of 
the  three  families  should  be  used  for  performing  the  normality  transformation.  A 
possible  procedure  is  to  compute  the  sample  estimate  of  the  standardized  moments, 
viz.,  the  coefficients  of  skewness  and  kurtosis,  and  choose  the  distribution  according  to 
which  of  the  two  regions  contains  the  computed  point.  However,  as  described  at  the 
beginning  of  the  chapter,  this  method  has  major  drawbacks.  Consequently,  another 
procedure  is  used  to  determine  the  family  of  distributions  to  be  used  to  perform  the 
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transformations.  It  is  a  simple  selection  rule  which  is  a  function  of  four  percentiles 
to  select  one  of  the  three  families  and  to  give  estimates  of  the  parameters  for  all  the 
families.  It  was  developed  by  Slifker  J.  and  Shapiro  S.  in  [4]. 

The  idea  of  the  selection  rule  is  to  try  and  find  a  property  of  the  transformation 
given  in  eq.(2.18)  and  use  it  to  select  an  appropriate  member  of  the  Johnson  family 
to  approximate  a  set  of  data.  It  was  heuristically  felt  by  Slifker  J.  and  Shapiro  S.  that 
there  must  be  some  relationship  concerning  the  distances  in  the  tails  vs.  distances 
in  the  central  portion  of  the  distribution  which  could  be  used  to  distinguish  between 
the  bounded  and  unbounded  cases.  This  led  to  the  following  formalization. 

Consider  any  one  of  the  transformations  described  by  eq.(2.18).  Choose  any  fixed 
value  of  r  >  0  of  a  standard  normal  variate.  Then  the  points  ±r  and  ±3r  divide  a 
horizontal  axis  into  three  intervals  of  equal  length  given  by  (—3/%  — r),  {—r,r),  (r,3r). 
Let  and  be  the  values  corresponding  to  3r^r,—r  and  —3r, under  the 

transformation  given  in  eq.(2.18),  respectively.  Let 


m  =  g3r  -  (Jr 

I  =  g-r-  9-zr  (2.23) 

V  =  Jr-g-r- 

Since  fi{g-,X,e),i  =  1,2,..., 3,  are  single-valued  monotonically  increasing  functions 
for  the  allowed  ranges  of  g,  it  is  readily  seen  that  m,  /  and  p  are  all  greater  than 
0.  For  a  bounded  symmetrical  Johnson  distribution,  it  was  hypothesized  that  the 
distances  m  and  /  between  each  of  the  outer  and  inner  points  would  be  smaller  than 
the  distance  p  between  the  two  inner  points.  The  converse  would  be  true  for  the 
unbounded  case.  This  led  to  the  following  more  general  result: 


(0 

(u) 

{Hi) 


^  >  1  for  any  Su  distribution- 
^  <  1  for  any  Sb  distribution-, 
^  =  1  for  any  Sl  distribution. 


(2.24) 


These  properties  are  proven  in  Appendix  A  and  can  be  used  to  discriminate  among 
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the  three  families. 

Selection  Procedure 

The  selection  procedure  consists  of  the  following  steps: 


1.  Choose  a  fixed  value  of  r  >  0.  This  choice  should  be  motivated 
by  the  number  of  data  points.  In  general,  for  moderate  sized 
data  sets,  a  value  of  r  less  than  1  should  be  chosen  [4].  Any 
choice  of  r  greater  than  1  would  make  it  difficult  to  estimate  the 
percentile  of  ±3r.  A  typical  choice  is  to  use  a  value  of  r  close  to 
0.5  such  as  0.524.  This  would  make  3r  =  1.572  and  these  points 
correspond  to  the  70'^  and  the  94.2'^  percentiles  of  the  standard 
normal  distribution,  respectively.  However,  the  larger  the  number 
of  data  points,  the  larger  the  value  of  r  that  can  be  selected.  In 
the  Ozturk  algorithm  r  is  chosen  to  be  0.775449. 

2.  Determine  from  a  table  for  the  normal  distribution  the  probability 
Pa  =  Pr{R  <  a),  where  a  is  taken  to  be  either  3r,  r,  — r  or  — 3r. 
For  example,  if  r  =  0.5  then  Po.s  =  Pr{R  <  0.5)  =  0.6915. 

3.  Determine  integer  values  of  ka  such  that 


n 

where 


(2.25) 


ka  =  [nPa  +  ^],  (2.26) 

[.]  denotes  the  closest  integer,  and  a  =  3r,  r,  — r-,  — 3r. 

4.  Obtain  n  observations  of  the  random  variable  G,  where  G  is  re¬ 
lated  to  the  random  variable  R  through  eq.(2.18).  Order  these 
observations  from  the  smallest  to  the  largest  and  denote  the 
ordered  observation  by  g^. 

5.  Let 


9a  =  (2.27) 
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where  a  =  3r,  r,  —  r,  —  3r.  The  connections  between  ga-,kai  and  Pa 
are  explained  in  Appendix  B. 

6.  From  the  values  of  ga  obtained  in  step  5,  compute  the  distances 
m,  /,  and  p  according  to  eq.(2.23). 

7.  Use  the  criteria  in  eq.(2.24)  to  select  the  appropriate  member  of 
the  family  of  distributions. 

Since  the  g‘ ’s  are  continuous  random  variables,  the  probability  is  zero  that  {ml/p"^)  = 
1  .  Thus,  choice  of  the  Sl  distribution  requires  that  mlfp^  fall  within  some  small 
prespecified  tolerance  interval  around  1. 

After  completion  of  the  selection  process,  the  next  step  is  to  estimate  the  parameters 
of  the  distribution  selected.  Estimation  of  the  parameters  is  accomplished  by  using 
the  formulae  given.  These  allow  the  estimates  to  be  simply  calculated  by  means  of 
a  scientific  hand  calculator.  The  formulae  for  the  estimates  are  given  in  terms  of 
the  chosen  values  of  r  and  the  computed  values  of  m,  I  and  p.  Derivations  of  these 
formulae  are  provided  in  Appendix  A. 

Note  that  the  following  formulae  express  the  parameter  values  as  functions  of  m,  I 
and  p  which  in  turn  are  functions  of  g3r,gr,g-r  and  g-3r-  In  practice,  the  corre¬ 
sponding  parameter  estimates  are  computed  based  on  the  ordered  sample  values, 
g'^o-  a  =  3r,  r,  — r,  — 3r. 

(i)  Johnson  Su  Distribution 


r 


=  7  +  7sinh  ^ 


Parameter  Estimates  for  Johnson  Su  Distribution 


7 

A 


2r 


cosh-^m+i)] 


=  7/  sinh 


1  r  1  /  m 
2  V  p  ■  p 

l_  _  m 

-ir  p _ p 


2(^1  _  1)1/2 


(7  +  ?-2)("  +  2)^^^ 


;  (7  >  0) 

]; 

;  (A  >  0) 


(2.28) 


(2.29) 
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_  g,  +  g-,  P(i  -  7) 

'  “  2  2(^  +  2-2)- 

(ii)  Johnson  Sb  Distribution 


r  =  7  +  ??  ln(  — — )•  (2-30) 

X  +  e-  g 

Parameter  Estimates  for  Johnson  Sb  Distribution 


V  = 

7  = 

A  = 

e  = 


cosh-‘(M(l  +  ^(l  +  f)l’'")’ 

2(^f-l) 


S.R  _  1  ’ 

m  / 

gr  -  g-r  _  A  p(?  -  i) 

2  2'^2(Jf-l)- 


(Hi)  Johnson  Si  Distribution 


r  =  7'  +  7/ln(5r  -  e). 
Parameter  Estimates  for  Johnson  Si  Distribution 


(2.31) 


(2.32) 


2r 

^  ^  hT^ 

P 

m  _ 

T''  = 

_  gr  +  g-r  _  1 

^  ~  2  22^-1' 

V 

Note  that  the  values  of  the  parameters  above  are  presented  in  such  a  way  as  to 
emphasize  their  dependence  on  the  ratios  m/p  and  Ijp  for  the  Su  distribution  and  on 
p/m  and  pjl  for  the  Sb  distribution  .  For  the  Si  distribution,  we  see  from  eq.(2.24) 
that  (//p)  =  {m/p)~^.  Thus,  the  formulae  for  the  Si  distribution  parameters  are 
given  solely  in  terms  of  the  single  ratio  m/p. 
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Resistor  Interval 
(in  kfi) 

Observed  Frequencies 

<9.25 

- 

9.25-9.75 

1 

9.75-10.25 

7 

10.25-10.75 

18 

10.75-11.25 

36 

11.25-11.75 

70 

11.75-12.25 

115 

12.25-12.75 

199 

12.75-13.25 

437 

13.25-13.75 

929 

13.75-14.25 

1787 

14.25-14.75 

2294 

14.75-15.25 

2082 

15.25-15.75 

1129 

15.75-16.25 

275 

16.25-16.75 

55 

16.75-17.25 

6 

>17.25 

- 

Total 

9440 

Table  2.1:  Resistor  Values 


Example: 

We  consider  a  set  of  data  representing  the  resistor  values  in  a  VLSI  circuit.  The 
data  and  the  observed  frequencies  are  shown  in  table  2. 3. 1.3.  We  choose  the  value  of 
r  to  be  1.  Thus,  the  four  values  assumed  by  a  are  +3,  1,-1  and  -3.  From  the  table 
for  the  normal  distribution,  the  probabilities.  Pa  =  Pr{R  <  a),  for  a  =  3, 1,  —1,  and 
—3  are  found  to  be  0.9986,0.8413,0.1587  and  0.0014,  respectively. 

First,  consider  a  =  3,  for  which  P3  =  0.9986.  The  value  of  the  order  number  ks  is 
given  by 

ks  =  [nP3  +  \]  =  [(9440)(0.9986)  +  0.5]  =  9427  (2.34) 

where  [.]  denotes  the  closest  integer  value.  If  the  raw  data  were  available,  we  would . 
simply  let  gs  equal  to  the  9427*^  ordered  sample,  However,  because  the  raw 

data  has  been  grouped  into  the  intervals  tabulated  in  table  2. 3. 1.3,  the  value  of 
is  unknown.  Consequently,  interpolation  is  used  to  estimate  a  value  for 

Note  that  the  9427*^  ordered  observation  falls  in  the  interval  (16.25,16.75).  The 
probabilities  that  the  resistor  values  are  less  than  or  equal  to  16.25kn  and  IQ.lbkil, 
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respectively,  are  given  by 


Pr{G  <  16.25)  =  ^  =  0.9935 

1/  iTv/ 

9440  -  6 

Pr(G<  16.75)  =  —9^^  =  0-9994. 


(2.35) 


Thus,  by  the  method  of  interpolation. 


16.75  -  16.25  ^  16.25 

0.9994  -  0.9935  0.9986  -  0.9935  ^  ’ 

The  value  of  is  found  to  be  16.25  +  0.439  =  16.689.  Setting  gs  equal  to  ,  it 
follows  that  gs  =  16.689.  Values  of  gi,g^i,  and  g^s  are  found  in  a  similiar  manner. 
In  summary. 


=  16.689 
=  15.242 
=  13.581 
=  10.409. 


(2.37) 


Consequently, 


P  =  gi-g-i  =  lMl 
m  =  5f3 -S'!  =  1-447 
/  =  g-i-g-3  =  3.172 


(2.38) 


yielding 


^  =  1.664. 


(2.39) 


Since  the  value  of  ^  is  found  to  be  significantly  greater  than  1,  it  is  decided  from 
eq.(2.24)  that  an  Su  distribution  is  appropriate  for  transformation.  The  formulae 
given  in  eq.(2.29)  are  used  to  obtain  the  parameter  values.  Thus, 
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cosh  ^[|(0. 871  +  1.910)] 


2.333 


1.910  -  0.871 


1  ^  1  402 

2^1.910(0.871)  -  1 

2(1.661)^(1.910)(0.871)  -  1 
(0.871  +  1.910  -  2)v/0.871  +  1.910  +  2 
15.242  +  13.581  ,  1.661(1.910  -  0.871) 

2  "^2(1.910  +  0.871-2) 


=  1.585 


=  15.516. 


(2.40) 


The  transformation  equation,  therefore,  becomes 

r  =  1.402  +  2.333  sinh-'(^———)  (2.41) 

1.585 

where  r  is  a  standard  normal  variable  and  g  is  a.  random  variable  corresponding  to 
the  resistors  values. 

Once  the  transformation  equations  have  been  obtained  for  the  end  point  coordinates 
Uon  and  Von,  they  are  applied  to  the  end  point  data  arising  from  the  2,000  Monte 
Carlo  simulations  to  generate  the  standard  bivariate  normal  random  variables  Rou 
and  respectively.  If  a  type  j  transformation,  j  =  1, 2, 3,  is  used,  the  original  data 
is  said  to  have  a  Johnson  type  j  distribution.  In  practice,  Uon  s^nd  Vo^  need  not  have 
the  same  distributions  (i.e.,  Uon  may  be  of  type  i  whereas  Von  may  be  of  type  j  and 
*  j)-  An  estimate  of  the  correlation  coefficient  between  Rqu  and  Roy  is  given  by 


1999  J 


(2.42) 


where  /iro„?  /lro„  and  o-rg^,  arg^  are  the  sample  means  and  variances  of  the  2,000 
transformed  statistics  ilou  and  Roy,  respectively. 

Since  Rqu  and  R^y  are  bivariate  standard  normal  random  variables,  their  joint  PDF 
can  be  written  as 


fRou,Rcv{rou,roy)  =  exp(-|) 


(2.43) 


where 
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t  =  ^  _  >2  (^o«  +  -  2^rouro„). 


(2.44) 


Let  t  =  to.  Then  the  equation 


<0  =  ^  _  ■;  (ro.  +  -  2^ro„ro„) 


(2.45) 


is  that  of  an  ellipse  in  the  rou,  f’ov  plane  for  which 


fficu,noArou,rov)  =  (2-46) 

Points  that  fall  within  the  ellipse  correspond  to  those  points  in  the  rou,rov  plane  for 


which 


f/ictL,^v(^0u,^0v)  >  p  2^' 


(2.47) 


Let  a  be  defined  as  the  probability  that  rou  and  rou  fall  outside  the  ellipse  given  that 
the  data  comes  from  the  null  hypothesis.  It  follows  that 


a  =  Pr{T  >  to). 


(2.48) 


Note  that  the  bivariate  normal  distribution  is  a  special  case  of  the  spherically 
invariant  random  vector  (SIRV)  where  the  characteristic  PDF  is  given  by  [11]. 


fs(^)  =S(s-  1). 


(2.49) 


The  PDF  of  an  N-dimensional  SIRV  involves  the  same  quadratic  form  t  that  arises 
in  the  N-dimensional  multivariate  Gaussian  PDF.  For  an  SIRV,  Rangaswamy  [11] 
shows  that  the  PDF  of  t  is 


Mt)  =  ^  ^hNity,  (0  <  f  <  oo) 

2^r(^) 

where  /iAr(<)  is  a  monotonically  decreasing  function  given  by 


(2.50) 


roo  f 

^N{t)-J^  s  exp-— /s(s)ds. 


Substituting  eqn.(2.49)  into  eq.(2.51),  there  results 


(2.51) 
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(2.52) 


f 

hN{t)  =  /  s"-''^exp(-— )(5(5  -  l)(/5 

Jo  zs^ 

=  exp(-|). 

For  the  bivariate  case,  N  —  2.  Consequently,  eq.(2.50)  reduces  to 

/t(^)  =  ^exp(-|);  0<t<oo.  (2.53) 

Hence, 

a  =  Pr{T  >  to)  =  iexp(-|)dt  =  exp(-y).  (2.54) 

Consequently,  to  =  — 21n(a).  Thus  eq.(2.45)  becomes 

; '1  >2'(^ou  +  ^ot,  -  2pro„ro„)  =  -2  ln(Q;).  (2.55) 

P 

The  contour  equation 

^lu  -  ‘^prouVov  +  =  -2(1  -  p^)  ln(Q:),  (2.56)  ■ 

which  is  the  equation  of  an  ellipse,  is  used  to  determine  the  100  x  (1  —  a)%  confidence 
contour.  This  is  also  shown  in  [15]  and  [16].  When  the  statistics  Rqu  and  Rov  are 
uncorrelated,  the  correlation  coefficient  is  0  and  eq.(2.56)  becomes 

+  = -21n(a),  (2.57) 

which  is  the  equation  of  a  circle.  Also,  eq.(2.56)  degenerates  into  a  line  as  the  corre¬ 
lation  coefficient  approaches  ±1. 

In  the  Ozturk  Algorithm,  an  inverse  Johnson  Transformation  is  applied  to  the  points 
for  the  confidence  ellipses.  The  locus  of  the  resulting  points  obtained  is  then  plotted 
to  obtain  the  corresponding  confidence  contours  in  the  U  —  V  plane.  Consequently, 
these  confidence  contours  are  not  necessarily  ellipsoidal. 

The  confidence  contours  are  plotted  for  a  given  sample  size  n.  These  are  then  used 
to  make  a  visual  as  well  as  computational  test  of  the  null  hypothesis.  If  the  terminal 
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point,  Qn,  of  the  sample  data,  falls  inside  the  contour,  the  data  is  declared  as  being 
consistent  with  the  null  hypothesis  with  confidence  level  (1  —  a).  Otherwise  the  null 
hypothesis  is  rejected  with  a  significance  level  a.  Fig.2.2  shows  the  linked  vectors 
and  the  confidence  contours  when  the  null  distribution  is  standard  normal  and  the 
sample  data  size  is  100.  From  the  figure  it  is  seen  that  the  sample  data  is  statistically 
consistent  with  the  null  hypothesis  at  confidence  level  of  90%.  Fig.2.3  shows  a  case 
where  the  sample  data  is  not  consistent  with  the  null  hypothesis  at  a  siginificance 
level  of  1%. 

2.3.2  Distribution  Approximation 

The  distribution  approximation  procedure  is  simply  an  extension  of  the  Goodness 
of  Fit  test.  Following  a  similiar  approach  to  that  outlined  in  the  section  for  the  Good¬ 
ness  of  Fit  test,  random  samples  are  generated  from  a  library  of  different  univariate 
probability  distributions.  In  the  Goodness  of  Fit  test,  the  statistic  Qon  =  {UoniVon) 
given  by  eq.(2.15)  was  obtained  for  the  null  hypothesis  and  for  a  specified  n.  For  the 
distribution  approximation  we  go  one  step  further  and  for  each  distribution  taken 
from  a  library  of  distributions,  we  obtain  the  end  point  statistic  Qn  from  eq.(2.12) 
for  a  given  n  and  for  various  choices  of  the  shape  parameter.  Thus,  depending  on 
whether  it  has  a  shape  parameter  or  not,  each  distribution  is  represented  by  a  point 
or  a  trajectory  in  a  two  dimensional  plane  whose  coordinates  are  [/„  and  14.  Fig.  2.4 
shows  an  example  of  such  a  representation.  The  distributions  which  are  plotted  on  the 
distribution  approximation  chart  are  (1)  Gaussian,  (2)  Uniform,  (3)  Exponential,  (4) 
Laplace,  (5)  Logistic,  (6)  Cauchy,  (7)  Extreme  Value,  (8)  Gumbel  type-2,  (9)  Gamma, 
(10)  Pareto,  (11)  Weibull,  (12)  Lognormal,  (13)  Student-T,  (14)  K-distributed,  (15) 
Beta,  and  (16)  Su-Johnson.  Tables  2.2  and  2.3  give  the  standard  and  the  general 
form  respectively,  of  these  distributions. 

vspace*8in 

vspace*8in 

As  mentioned  before,  the  points  on  the  linked  vectors  for  various  distributions  are 
computed  using  eq.(2.12).  The  magnitude  for  each  point  on  the  linked  vectors  is 
computed  from  values  averaged  over  2,000  Monte  Carlo  simulations  of  the  ordered 
statistic,  Yi..n,  obtained  from  eq.(2.6)  while  the  angles  are  computed  from  the  reference 
distribution  (standard  Gaussian).  The  confidence  ellipses  are  computed  only  for  the 
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Figure  2.2:  The  Confidence  Contours  and  the  linked  vectors  with  standcird  normril  as  null 
Dotted  Line  =  Null  Distribution  Pattern,  Dashed  Line  =  Sample  Distribution  Pattern 
90,  95,  99%  contours  from  the  innermost  to  the  outermost  respectively. 


Distribution 


Gaussian 


Uniform 


Exponential 


Laplace 


Logistic 


Cauchy 


Extreme  Value  (Type  1) 


Gumbel  (Type  2) 


Gamma 


Pareto 


Weibull 


Lognormal 


K-Distribution 


Beta 


Johnson-SU 


_ Standard  Form  fyiv) 


(■V^^)  ^  exp(- V)  -  oo  <  y  <  oo 


1  0  <  y  <  1 


exp(-y)  0  <  y  <  oo 


0.5exp(— |y|)  -  oo  <  y  <  oo 


exp(-y) 


•  —  oo  <  y  <  oo 


exp(-y)  exp[-  exp(-y)]  -  oo  <  y  <  oo 


7yexp(-7  -  l)exp(-y~'^)  oo  <  y  <  oo 


exp(-y)y'>'“^  a  <  y  <  oo 


y  >  1,7  >  0 


7y'^  ^exp(-y^)  y>0 


exp[-i^‘°fy))  ]  y  >  0 


(|)'^A%-iy  y>0 


o<y<l 


■y2)ri-v/l  +  S/* 


—  oo  <  y  <  OO 


Table  2.2:  Standard  Forms  of  the  PDF’s  used  in  the  Approximation  Chart 


Distribution 


Gaussian 


Uniform 


Exponential 


Laplace 


Logistic 


Cauchy 


Extreme  Value  (Type  1) 


Gumbel  (Type  2) 


Gamma 


Pareto 


Weibull 


Lognormal 


K-Distribution 


Beta 


Johnson-SU 


General  Form  fxix) 


Table  2.3;  General  Forms  of  the  PDF’s  used  in  the  Approximation  Chart 
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null  hypothesis  used  in  the  prior  Goodness  of  Fit  test.  Only  the  end  point  coordinates 
Qn  of  the  linked  vectors  are  provided  in  the  approximation  chart.  This  is  due  to  the 
fact  that  the  plot  would  become  too  cluttered  to  properly  interpret  the  data  if  all 
the  linked  vectors  for  these  various  distributions  were  provided  in  the  graphics.  Also, 
meaningful  information  from  the  linked  vectors  is  contained  in  the  location  of  their 
end  points.  Therefore,  only  the  end  points  of  all  the  linked  vectors  are  provided 
in  the  approximation  chart,  along  with  the  confidence  ellipses  for  the  selected  null 
distribution. 

For  each  distribution,  such  as  Gaussian,  that  is  uniquely  specified  by  its  mean  and 
variance  (no  shape  parameters),  there  exists  a  single  end  point  on  the  approximation 
chart  corresponding  to  the  single  unique  linked  vector. 

For  distributions  dependent  on  a  single  shape  parameter,  such  as  Weibull,  the  end 
point  of  the  of  the  linked  vector  will  also  be  dependent  on  the  shape  parameter. 
Therefore,  a  sequence  of  linked  vectors  must  be  computed  in  order  to  obtain  the 
trajectory  on  which  the  end  point  travels  for  varying  shape  parameter.  In  a  sense,  the 
trajectory  represents  a  family  of  PDFs  having  the  same  distribution  but  with  different 
shape  parameter  values.  For  example,  the  trajectory  for  the  Weibull  distribution  is 
obtained  by  joining  the  end  points  for  which  the  shape  parameters  are  0.3,  0.4,  0.5, 
0.6,  0.8,  1.1,  1.5,  2.0,  3.0,  5.0.  As  the  shape  parameter  increases,  note  that  the 
Weibull  distribution  approaches  the  Gaussian  distribution.  This  is  shown  in  fig.  2.4. 
The  representation  of  fig.  2.4  is  called  an  approximation  chart. 

Similiarly,  for  a  distribution  dependent  on  two  shape  parameters,  such  as  Beta,  a 
sequence  of  linked  vectors  must  be  computed  in  order  to  plot  the  trajectories  on  which 
the  end  point  travels  for  varying  shape  parameters.  This  is  performed  by  holding  the 
first  shape  parameter  constant  and  varying  the  second  shape  parameter  to  generate 
a  trajectory,  then  changing  the  first  shape  parameter  and  again  holding  it  constant 
while  varying  the  second  shape  parameter,  etc...  until  a  family  of  trajectories  is 
produced  that  characterizes  the  distribution. 

Thus,  an  approximation  chart  such  as  that  in  fig.  2.4  can  be  produced.  It  is 
apparent  that  this  approximation  chart  provides  a  one  to  one  graphical  representation 
for  each  PDF  for  a  given  n.  Therefore,  every  point  in  the  approximation  chart 
correponds  to  a  specific  distribution.  Thus,  if  the  null  hypothesis  in  the  Goodness  of 
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Figure  2.4;  The  Approximation  Chart.  1)  N  =  Normal,  2)  U  =  Uniform,  3)  C  =  Cauchy, 
4)  L  =  Lognormal,  5)  S  =  Logistic,  6)  A  =  Laplace,  7)  V  =  Extreme  Value,  8)  T  =  T2- 
Gumbel,  9)  G  =  Gamma,  10)  E  =  -ve  Exponential,  11)  P  =  Pareto,  12)  K  =  K-Distributed, 
13)  W  =  Weibull,  14)  B  =  Beta,  15)  SU  =  SU-Johnson. 


Fit  test  is  rejected,  then  the  distribution  which  approximates  the  underlying  PDF  of 
the  set  of  random  data  can  be  obtained  by  comparing  Qn  obtained  for  the  samples 
with  the  existing  trajectories  in  the  chart.  The  end  point  or  trajectory  closest  to 
the  Qn  of  the  sample  data  is  chosen  as  an  approximation  to  the  PDF  underlying  the 
random  data.  This  closest  point  or  trajectory  is  determined  by  projecting  the  sample 
point  Qn  to  neighbouring  points  or  trajectories  on  the  chart  and  selecting  that  point 
or  trajectory  whose  perpendicular  distance  from  the  sample  point  is  the  smallest.  For 
example  consider  the  situation  of  fig.  2.5.  Let  Qn  —  {un,Vn)  denote  the  coordinates  of 
the  sample  point.  Let  (xi,?/i)  and  (a;2,j/2)  denote  the  coordinates  of  the  points  A  and 
B  on  the  trajectory  shown  in  fig.  2.5.  The  segment  of  the  trajectory  between  points 
A  and  B  is  assumed  to  be  linear.  Let  (a;o,i/o)  denote  the  coordinates  of  the  point  of 
intersection  of  the  straight  line  between  A  and  B  and  the  projection  of  Qn  = 
onto  this  straight  line.  The  equation  of  the  straight  line  between  the  points  A  and  B 
can  be  written  as 


y  —  yi  =  rn(x  —  xi)  (2.58) 

where  m  =  and  {x,y)  is  a  point  on  the  line.  Also,  the  equation  of  the  straight 

line  joining  (xo,2/o)  and  (un,u„)  is 

y  -  Vn-  — ^(x  -  Un)  (2.59) 

m 

where  (x,y)  is  a  point  on  the  perpendicular.  The  coordinates  (xo,yo)  result  from 
letting  X  =  xq  and  y  =  yo  in  eqs.(2.58)  and  (2.59).  Their  solution  yields 


xo  =  -— ■■■  -  m?ji  +  +  mvn] 

+  I 

yo  =  . -^—[yi  -  mxi  +  m2Vn  +  mUn].  (2.60) 

+  1 

Finally,  the  perpendicular  distance  from  the  sample  point  onto  the  trajectory  between 
points  A  and  B  is 

^  \  -  2mV’i?/’2  +  ^2]  (2-61) 

V  +  1 
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where 


V’l  =  «n  -  Xi, 

tl’2  =  Vn-yi.  (2.62) 

The  complete  distribution  approximation  algorithm  is  summarized  as  follows. 

1.  Sort  the  samples  Xi ,  X2., . . . ,  Xn  in  increasing  order. 

2.  Obtain  the  standardized  order  statistic  Yi:n- 

3.  Compute  Un  and  14  from  eq.(2.12)  for  the  library  of  PDFs  men¬ 
tioned. 

4.  Obtain  an  approximation  chart  based  on  the  sample  size  n  and 
plot  the  sample  point  Qn  on  this  chart. 

5.  Compute  the  distance,  D  between  the  sample  point  Qn  and  each 
of  the  end  points  on  the  chart.  Choose  the  PDF  corresponding  to 
the  point  or  trajectory  that  results  in  the  smallest  value  for  D  as 
an  approximation  to  the  PDF  of  the  samples. 

The  approximation  to  the  underlying  PDF  of  the  set  of  random  data  can  be  im¬ 
proved  by  including  as  many  distributions  as  possible  in  the  approximation  chart  so 
as  to  fill  as  much  of  the  space  as  possible  with  candidate  distributions.  It  is  empha¬ 
sized,  however,  that  this  procedure  does  not  identify  the  underlying  PDF.  It  merely 
gives  the  best  approximation  to  the  distribution  underlying  the  PDF  of  the  data  from 

those  available  in  the  chart. 

2.3.3  Parameter  Estimation 

Once  the  probability  distribution  of  the  samples  is  approximated,  the  next  step  is 
to  estimate  its  parameters.  The  method  of  distribution  approximation  discussed  in 
2.3.2  lends  itself  for  estimating  the  parameters  of  the  approximated  distribution.  We 
present  the  estimation  procedure  for  the  location,  scale  and  shape  parameters  in  this 
section. 
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2. 3. 3.1  Estimation  of  Location  and  Scale  Parameters 

Let  f{x  :  a,/3)  denote  a  known  distribution  which  approximates  the  PDF  of  the 
set  of  random  data,  where  a  and  /?  are  the  location  parameter  and  scale  parameter, 
respectively,  of  the  approximating  PDF.  Let  Xi^n  denote  the  ordered  statistics  of  X 
from  a  sample  of  size  n.  Let  Si-.n  be  defined  by 


= 


Xi-n 


Also,  let 


/^t:n  —  . 


Then 


E[Xi,n]  =  ^Hi:n  +  Ol. 


We  consider  the  following  statistics 


T\  =  '^COs{6i)Xi,n 


(2.63) 


(2.64) 


(2.65) 


T2  =  ^sin(0,)A^:„ 


(2.66) 


where  Oi  is  the  angle  defined  in  eq.(2.10).  The  expected  values  of  Ti  and  Tj  are 


^[^1]  = 


^[^2]  =  X]  sin(6li)  [/?;«,•:„  + q]. 


(2.67) 


These  can  be  written  as 


jE[Ti]  =  aa  + 
E[T2]  =  ca-{-  d  13 


(2.68) 
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where 


a  =  ^cos(<?,) 

i=l 

n 

^  cos(0,) 

i=l 

c  =  ^sin(0,)  (2.69) 

t=i 

n 

i=l 

Because  the  standardized  Gaussian  distribution  is  used  as  the  reference  distribution 
for  6>i,  it  can  be  shown  that  a  =  0  [1].  The  estimates  for  3  and  a  are  then  given  by 


E[T2]-d^ 

c 


(2.70) 


For  sufficiently  large  n  (i.e.,  n  >  50),  suitable  estimates  for  jEfTi]  and  £^[22]  are 


E[Ti]  «  Ti 

£;'IT2]  T2.  (2.71) 

Estimates  for  b  and  d  rely  upon  an  estimate  of  is  obtained  from  a  Monte  Carlo 

simulation  of  Si,n  where  5,:„  is  generated  from  the  known  approximating  distribution 
/(x;  0, 1)  having  zero  location  and  unity  scale  parameters,  fii-.n  is  the  sample  mean  of 
5,-.„  based  upon  2,000  Monte  Carlo  trials.  Having  fn-.n,  the  estimates  for  b  and  d  are 
given  by 

n 

^  A.:nCOs(6>i) 

d  =  ^/i,:„sin(0,).  (2.72) 

i=l 
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The  scale  and  location  parameters  are  then  estimated  by  application  of  eqs.(2.70)  and 
(2.71). 

2. 3. 3. 2  Shape  Parameter  Estimation 

In  this  section  we  present  the  approximate  method  used  for  estimating  the  shape 
parameter  of  the  approximating  PDF.  We  first  consider  distributions  with  only  one 
shape  parameter.  Let  7  denote  the  shape  parameter  of  the  approximating  PDF.  Since 
Un  and  Vn  are  location  and  scale  invariant,  the  point  depends  only  on  the  sample 
size  n  and  the  shape  parameter  7. 

Recall  that  a  point  on  the  trajectories  of  the  approximation  chart  is  obtained  by 
averaging  for  a  specified  value  of  the  shape  parameter  the  results  from  a  large  number 
of  trials  for  t/„  and  Consequently,  for  given  values  of  n  and  7  the  coordinates, 
of  the  corresponding  point  along  the  trajectory  for  a  specified  distribution  can  be 
characterized  by 


E{Ur>)  = 

E{Vn)  =  4>2{nri)  (2.73) 


where  the  complete  trajectory  is  obtained  by  repeating  the  large  number  of  trials 
for  Un  and  Vn  over  a  suitable  range  of  7.  On  a  given  trial  involving  the  random 
data  it  is  likely  that  the  coordinates  f/„  and  K,  obtained  for  the  samples  will  not 
coincide  with  any  of  the  trajectories  on  the  chart.  The  PDF  underlying  the  random 
data  is  approximated  by  selecting  the  distribution  corresponding  to  the  point  in  the 
trajectory  that  falls  closest  to  the  sample  point  Qn-  The  situation  is  illustrated  in 
fig.  2.6.  Qn  appears  in  the  figure  with  coordinates  {Un,Vn).  The  straight  line  fr’ 
denotes  an  approximation  to  a  segment  of  the  nearest  trajectory  which,  in  general,  is 
a  curved  segment  between  points  A  and  B.  A  is  that  point  on  the  actual  trajectory 
corresponding  to  the  shape  parameter  7^.  Its  coordinates  are  {ua,va)-  Similiarly,  B 
is  the  point  on  the  actual  trajectory  corresponding  to  the  shape  parameter  7^.  Its 
coordinates  are  {ub,vb)-  The  slope  of  the  straight  line  between  points  A  and  B  is 


vb  -  va 

m  =  - . 

UB  -  ua 


(2.74) 
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The  equation  for  the  straight  line  Tr  is 


V  —  va  +  —  ua)-  (2.75) 

Point  C,  with  coordinates  (uc,  Wc),  is  the  perpendicular  projection  of  Qn  onto  Tr.  The 
straight  line  linking  Qn  and  C  has  a  slope  equal  to  — and  an  equation  of  the  form 

t;  =  V;--(m-C/„).  (2.76) 

m 

Since  C  is  a  point  common  to  both  straight  lines,  it  follows  from  eqs.(2.75)  and  (2.76) 
that 


Va  +  m{uc  -  Ua)  =  Vn - {uc  -  Un). 

m 


(2.77) 


Solution  for  uc  results  in 


7n(  Vn  -  •^■4)  +  "mu A  +  Un 
+  1 


(2.78) 


Let  7c  denote  the  shape  parameter  corresponding  to  the  point  on  the  actual  trajectory 
closest  to  Qn.  An  approximation  to  7c  is  then  obtained  by  linear  interpolation  on 


T,.  The  result  is 


7c  =  7  a  + 


(7b  -  7a){uc  -  Ua) 


(2.79) 


\  '  S'''/ 

(UB  -  Ua) 

The  accuracy  of  the  procedure  can  be  improved  by  employing  a  non-linear  interpo¬ 
lation  method.  It  must  be  emphasized  that  the  location,  scale  and  shape  parameter 
estimation  procedures  presented  in  this  section  are  approximate  procedures. 

The  proposed  estimation  procedure  can  also  be  extended  to  the  two  -shape  param¬ 
eter  case.  In  this  case  one  needs  to  choose  at  least  three  points  (ui,  ui),  (u2)  *^2)  and 
(u3,  V3)  and  let  the  shape  parameter  values  corresponding  to  these  three  points  be 
71,72  and  73,  respectively.  The  points  are  chosen  in  such  a  way  that  they  form  the 
three  vertices  of  a  triangle  inside  which  falls  the  sample  point  Q„  [1].  Again,  by  using 
a  linear  interpolation  in  the  plane,  an  approximate  solution  can  be  obtained  for  the 
parameter  estimates. 
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Chapter  3 

Simulation  Results  of  the  Ozturk 
Algorithm 


For  univariate  cases,  the  power  of  the  Ozturk  Algorithm  has  been  studied  for  various 
distributions  in  [1],  [2]  and  [3].  It  was  noted  in  [1]  through  [3]  that  the  power  of  the 
algorithm  depends  on  the  sample  size  n,  type  of  the  standardized  statistic  and  the 
null  distribution.  This  algorithm  has  been  found  to  compare  favorably  against  all  the 
well  known  tests.  Also,  the  algorithm  has  been  put  to  use  to  test  its  performance 
against  different  known  distributions.  Random  data  were  generated  using  computer 
simulations  as  given  in  [18]  and  [19].  The  Goodness  of  Fit  test  as  well  as  the  Dis¬ 
tribution  Approximation  test  was  performed  on  these  data  using  this  Algorithm.  In 
this  chapter,  a  brief  summary  of  some  of  the  results  obtained  is  presented. 

Data  was  generated  from  four  different  null  distributions,  viz., 

•  Univariate  Gaussian 

•  Weibull  (Shape  Parameter  1) 

•  Gamma  (Shape  Parameter  1) 

•  Lognormal  (Shape  Parameter  1). 

The  Goodness  of  Fit  test  results  are  tabulated  and  presented  first.  The  results  of 
the  Distribution  Approximation  are  not  easy  to  tabulate.  We  shall,  therefore,  present 
the  result  of  a  single  case  for  the  purpose  of  illustration. 
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3.1  Goodness  of  Fit  Test  Results 

3.1.1  The  Univariate  Gaussian  Case: 

Data  was  generated  from  a  Gaussian  pseudo  random  number  generator  using  com¬ 
puter  simulations.  The  data  set  represented  a  zero  mean  and  unit  variance  normal 
PDF.  The  following  observations  were  noted. 

•  It  was  observed  that  a  sample  size  of  less  than  40  is  not  advisable 
for  the  Goodness  of  Fit  test  as  it  almost  always  shows  that  the 
data  is  statistically  consistent  with  any  null.  This  is  due  to  the 
fact  that  such  a  small  sample  size  could  be  used  to  represent  any 
PDF. 

•  A  sample  size  between  75  and  100  is  found  to  be  good  enough  to 
accurately  perform  the  Goodness  of  Fit  test. 

•  For  a  sample  size  greater  than  75  and  when  the  null  specified  was 
Gaussian,  the  Goodness  of  Fit  test  showed  that  the  data  was  sta¬ 
tistically  consistent  with  the  null  in  almost  all  the  cases  .  For  other 
null  hypotheses  which  are  not  close  to  Gaussian  in  the  approxima¬ 
tion  chart,  the  Goodness  of  Fit  test  always  showed  that  the  data 
was  statistically  inconsistent  with  the  nulls.  But  for  null  hypothe¬ 
ses  which  are  close  to  Gaussian  in  the  approximation  chart,  such 
as  logistic,  the  Goodness  of  Fit  test  comes  up  with  statistical  con¬ 
sistency  almost  always.  This  vindicates  the  fact  that  the  logistic 
PDF  curve  is  very  similiar  to  the  Gaussian  PDF  curve. 

Table  3.1  shows  the  results  obtained  for  this  case. 

3.1.2  The  Weibull  Case: 

Data  was  generated  from  the  Weibull  PDF  with  shape  parameter  1  and  the  Good¬ 
ness  of  Fit  test  was  performed  on  it.  The  following  observations  were  noted. 

•  The  Goodness  of  Fit  test  worked  well  for  a  sample  size  between 
75  and  100.  For  a  smaller  sample  size,  the  Goodness  of  Fit  test  is 
not  advisable  since  the  results  obtained  were  not  accurate. 
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Sample  Size 

Data  Generated 

Null 

No.  of  Cases  of  SC/ 

(n) 

From 

Distribution 

Total  No. of  Cases 

5 

Gaussian 

Gaussian 

3/3 

25 

3/3 

40 

4/7 

50 

4/7 

75 

7/7 

100 

7/8 

125 

8/8 

150 

8/8 

25 

Uniform 

2/5 

40 

2/5 

50 

1/5 

75 

0/5 

100 

0/5 

150 

0/5 

75 

Exponential 

0/8 

100 

0/8 

150 

0/8 

75 

Laplace 

4/8 

100 

2/8 

150 

1/8 

75 

Logistic 

7/8 

100 

7/8 

Table  3.1;  Results  of  the  Ozturk  Algorithm  when  the  data  generated  was  Gaussian.  SC 
indicates  Statistical  Consistency 
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Sample  Size 

Data  Generated 

Null 

No.  of  Cases  of  SC/ 

(n) 

From 

Distribution 

Total  No. of  Cases 

5 

Weibull  (Sh.l) 

Weibull  (Sh.l) 

7/8 

25 

7/9 

40 

4/8 

50 

5/8 

75 

“ 

6/8 

100 

8/8 

150 

8/8 

5 

Weibull  (Sh.l) 

Weibull  (Sh.2) 

7/8 

25 

3/8 

40 

2/8 

50 

0/8 

75 

0/9 

100 

0/8 

150 

0/8 

Table  3.2;  Ozturk  Algorithm  Results  when  data  generated  was  Weibull  with  Shape  Pa¬ 
rameter  1.  SC  indicates  Statistical  Consistency  and  Sh.  indicates  Shape  Parameter. 

•  When  the  null  specified  was  Weibull  with  shape  parameter  2,  the 
Goodness  of  Fit  test  showed  that  the  data  was  statistically  incon¬ 
sistent  with  the  null  for  a  sample  size  greater  than  75  in  all  the 
cases.  This  is  due  to  the  fact  that  Weibull  (Shape  Parameter  1)  is 
far  away  from  Weibull  (Shape  Parameter  2)  on  the  approximation 
chart. 

Table  3.2  shows  the  results  obtained  for  this  case. 

3.1.3  The  Gamma  Case: 

Data  was  generated  from  the  Gamma  PDF  with  shape  parameter  1  and  the  Good¬ 
ness  of  Fit  test  was  performed  on  it.  Observations  noted  were  almost  the  same  as 
those  for  the  Weibull  case.  Again  a  sample  size  between  75  and  100  was  observed  to 

have  performed  well  in  this  case.  The  results  for  this  case  are  tabulated  in  table  3.3. 

3.1.4  The  Lognormal  Case: 

A  Lognormal  pseudo  random  number  generator  was  used  to  generate  random  data 
representing  the  Lognormal  PDF  with  a  shape  parameter  of  1.  Observations  noted 
for  the  Goodness  of  Fit  test  performance  on  this  data  were  noted. 

•  A  sample  size  between  50  and  75  was  found  to  be  sufficient  to 
perform  the  Goodness  of  Fit  test  accurately. 
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Sample  Size 

Data  Generated 

Null 

No.  of  Cases  of  SC/ 

(n) 

From 

Distribution 

Total  No. of  Cases 

40 

Gamma  (Sh.l) 

Gamma  (Sh.l) 

5/7 

50 

7/8 

75 

7/9 

100 

7/8 

150 

8/8 

50 

Gamma  (Sh.l) 

Gamma  (Sh.5) 

1/8 

75 

0/8 

100 

1/8 

Table  3.3:  Ozturk  Algorithm  Results  when  data  generated  was  Gamma  with  Shape  Pa¬ 
rameter  1.  SC  indicates  Statistical  Consistency  and  Sh.  indicates  Shape  Parameter. 


Sample  Size 
(n) 

Data  Generated 
From 

Null 

Distribution 

No.  of  Cases  of  SC/ 
Total  No. of  Cases 

40 

Lognormal  (Sh.l) 

Lognormal  (Sh.l) 

5/8 

50 

7/8 

75 

8/8 

100 

8/8 

150 

8/8 

40 

Lognormal  (Sh.l) 

Lognormal  (Sh.0.5) 

0/9 

50 

0/8 

75 

0/8 

100 

0/8 

Table  3.4:  Ozturk  Algorithm  Results  when  data  generated  was  Lognormal  with  Shape 
Parameter  1.  SC  indicates  Statistical  Consistency  and  Sh.  indicates  Shape  Parameter. 


•  This  sample  size  gave  very  good  results  as  far  as  the  distribution 
approximation  was  concerned.  About  30  times  out  of  40  the  Log¬ 
normal  PDF  showed  up  in  the  five  closest  distributions  that  could 
be  approximated. 

Table  3.4  shows  the  results  of  the  Goodness  of  Fit  test  for  this  case. 

In  general,  the  Goodness  of  Fit  test  seemed  to  perform  well  for  a  sample  size  of  100. 
The  confidence  contours  grow  smaller  when  the  sample  size  is  increased.  In  effect,  we 
could  hypothesize  that  for  an  infinite  sample  size  the  contours  would  become  a  point 
in  the  two  dimensional  (17,  V)  plane.  This  is  intuitively  satisfying. 
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Distribution  No. 

Distance 

Rank 

21 

0.23728E-07 

1 

22 

0.89802 E-05 

2 

20 

3 

23 

0.28047E-04 

4 

5 

0.47542E-04 

5 

Table  3.5:  Five  closest  PDF’s  given  by  distribution  approximation  test  for  a  standard 
Gaussian  data  set 


Distribution  No. 

Scale 

Parameter 

Shape  (1) 
Parameter 

Shape  (2) 
Parameter 

21 

1.9774 

2.4099 

-0.2 

22 

-0.14976 

1.9392 

2.3697 

20 

-0.39697 

1.9747 

2.4310 

-0.4 

23 

1.9437 

2.3717 

0.0 

5 

-0.06215 

0.49795 

0.0 

Table  3.6:  Estimates  of  the  Parameters  of  the  five  closest  distributions  chosen  by  the 
distribution  approximation  test  for  a  standard  Gaussian  data 


3.2  Distribution  Approximation  Test  Results 

The  distribution  approximation  test  was  performed  for  a  number  of  cases.  In  fact, 
it  was  performed  for  all  the  cases  in  which  the  Goodness  of  Fit  test  was  performed. 
As  mentioned  previously,  since  it  is  not  very  easy  to  tabulate  the  results  of  the 
distribution  approximation  test  for  all  these  cases,  results  for  a  single  test  case  are 
presented  below. 

Data  was  generated  from  standard  Gaussian  distribution  using  a  Gaussian  random 
number  generator.  A  single  test  case  consisting  of  100  data  points  was  considered. 
Using  standard  Gaussian  as  the  null  distribution,  the  distribution  approximation  test 
was  performed  on  the  data  set.  The  first  results  of  this  test  gave  the  five  closest 
PDF’s  that  the  data  could  approximate.  This  result  is  shown  in  table  3.5 
Distributions  20,  21,  22  and  23  are  all  SU-Johnson  distributions  with  different  shape 
parameters,  where  as  distribution  number  5  is  logistic  distribution.  Note  that  the- 
standard  Gaussian  was  the  11th  ranked  PDF  with  a  distance  of  0.47879E-03.  Es¬ 
timates  of  the  location,  scale  and  the  shape  parameters  given  by  the  distribution 
approximation  test  for  these  distributions  are  given  in  table  3.6. 

The  approximation  chart  for  this  test  case  is  shown  in  fig.  3.1.  It  is  obvious 
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from  the  approximation  chart  that  the  PDF’s  identified  for  this  case  are  very  close 
to  the  Gaussian  PDF.  It  is  therefore  concluded  that  even  though  this  data  set  has 
passed  the  Goodness  of  Fit  test  with  standard  Gaussian  as  the  null,  they  could  also 
be  approximated  by  the  set  of  5  PDF’s  identified  in  table  3.5.  In  fact  these  are 
better  approximations  than  standard  Gaussian.  This  is  shown  by  histogram  plots 
shown  in  figures  3.2,  3.3  and  3.4.  In  these  plots  the  histogram  of  the  data  is  plotted 
along  with  the  null  hypothesis,  which  is  the  standard  Gaussian,  and  one  of  the  five 
distributions,  given  by  the  distribution  approximation  test,  on  the  same  coordinate 
axes.  As  is  obvious  from  the  figures  there  is  very  little  to  choose  amongst  the  five 
PDF’s  approximated. 
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Histogram  Fitting 


Magnitude 


Histogram  Fitting 


Magnitude 


Figure  3.2:  Histogram  Plot:  l)Histogram  plotted  for  the  data,  2)Dotted  curve  is  the 
standard  Gaussian,  3)Dashed  curve  is  PDF  no.  21  for  the  top  plot  and  PDF  no  22  for  the 
bottom  plot.  Parameters  of  the  PDFs  21  and  22  are  given  in  table 


I'rcqcncy  I'rcqcncy 


Histogram  Fitting 


Magnitude 


Histogram  Fitting 


Magnitude 


Figure  3.3:  Histogram  Plot:  l)Histogram  plotted  for  the  data,  2)Dotted  curve  is  the 
standard  Gaussian,  3)Dashed  curve  is  PDF  no.  20  for  the  top  plot  and  PDF  no  23  for  the 
bottom  plot.  Parameters  of  the  PDFs  20  and  23  are  given  in  table 
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Chapter  4 

Conclusions  and  Suggestions  for 
Future  Work 


4.1  Conclusions 

This  thesis  has  discussed  various  techniques  for  analyzing  random  data.  Two  areas 
were  considered.  The  first  area  dealt  with  Goodness  of  Fit  tests  to  determine  whether 
or  not  a  set  of  random  data  is  statistically  consistent  with  a  prespecified  probability 
distribution.  After  reviewing  the  Kolmogorov-Smirnov  test,  Chi-Square  test,  Q-Q 
Plots  and  P-P  Plots,  a  new  test,  called  the  Ozturk  Algorithm,  was  introduced.  This 
test  has  an  easily  understood  graphical  presentation  and  works  well  for  sample  sizes 
as  small  as  100.  The  second  area  dealt  with  approximation  of  the  underlying  PDF  of 
random  data.  Although  the  other  tests  were  not  applicable  to  this  second  area,  the 
Goodness  of  Fit  test  of  the  Ozturk  Algorithm  was  shown  to  lend  itself  to  generation  of 
a  distribution  approximation  chart  from  which  approximations  to  the  underlying  PDF 
of  the  random  data  can  be  obtained.  Again,  good  results  were  observed  for  sample 
sizes  as  small  as  100.  An  analysis  was  provided  for  generating  confidence  contours 
when  the  random  data  was  non  Gaussian.  Simulated  data  was  used  to  evaluate 
performance  of  the  Ozturk  Algorithm  and  some  of  the  results  were  presented. 

4.2  Suggestions  for  Future  Work 

Several  problems  remain  to  be  explored  with  the  Ozturk  Algorithm; 

1.  The  Ozturk  Algorithm  works  well  for  continuous  probability  den¬ 
sity  functions.  Generalizations  of  the  Ozturk  Algorithm  to  the 
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discrete  case  should  be  explored. 

2.  Extension  of  the  Ozturk  Algorithm  from  univariate  to  multivariate 
PDF’s  should  be  considered.  One  possibility  involves  utilization 
of  quadratic  forms  of  the  data  [7],  [11]. 

3.  Rangaswamy  [11]  demonstrated  that  multivariate  spherically  in¬ 
variant  random  processes  (SIRP’s)  can  be  approximated  by  means 
of  their  quadratic  forms.  However,  a  probability  distribution  ap¬ 
proximation  chart  for  SIRP’s  that  could  be  utilized  by  the  Ozturk 
Algorithm  remains  to  be  generated. 

4.  The  univariate  PDF’s  currently  included  in  the  approximation 
chart  are  unimodal.  Extension  to  multimodal  PDF’s  should  be 
explored. 

5.  When  the  number  of  data  points  is  much  greater  than  100,  the 
Ozturk  Algorithm  requires  considerable  time  to  process  the  data. 
Ways  should  be  examined  for  making  the  algorithm  more  efficient. 
This  includes  parallelization  of  the  algorithm  as  well  as  processing 
the  data  in  groups  of  100  and  averaging  the  results. 

6.  Reduction  of  the  Ozturk  Algorithm  to  chip  form  should  be  inves¬ 
tigated  for  real-time  applications. 


70 


Appendix  A 

Algebraic  Derivations  for  Johnson 
Distributions 


In  this  appendix,  the  criteria  given  in  ec|.(2.25)  of  chapter  2  are  established  and  the 
parameter  estimates  given  in  equations  (2.30),  (2.32)  and  (2.34)  are  developed. 

A.l  Johnson  Su  Distributions 

The  transformation  for  the  Johnson  Su  Distribution  is  of  the  form 

R  =  1  +  ri  sinh~^(  (A.l) 

where  jR  is  a  standard  normal  variable  and  e  is  a  location  parameter,  A  is  a  scale 
parameter,  and  7  and  77  are  shape  parameters  for  the  PDF  of  the  Su  Distribution. 
Solving  eq.(A.l)  for  G  in  terms  of  /?,  thei'e  results 

G==e  +  Asinh( - ^)  =  e  —  A  sinh(-^^ - )  (A. 2) 

77  77 

where  we  have  made  use  of  the  fact  that  sinh(y4)  is  an  odd  function  of  A.  Define 


m  -  gzr  -  9r 

1  =  9-r  -  9r  (A.3) 

P  =  9r-  9-t 

where  g±r,g±3r  are  obtained  from  eq.(A.2)  for  ±7”  and  ±3r,  respectively. 

For  a  fixed  positive  value  of  r,  eqs.(A.2)  and(A.3)  give 
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m  =  A[sinh(^^)-sinh(^— ^)] 

V  V 

=  A[sinh(^-^’'+’')-sinh(^-^’~ -'•)], 
V  V 

I  =  A[sinh(^^^^)-Sinh(^!^)] 

n  1 

xr  •  i_.7  +  2r +  r^  .  ,  ^j  +  2r 

=  A[sinh( - )  —  smh( - )J, 

Tf  77 

p  =  A[sinh(^^)-smh(^^)]. 

V  V 


(A.4) 


Using  the  standard  formula 


sinh(  A  +  B)  —  sinh(A  —  B)  —  2  cosh  A  sinh  B, 


(A,6) 


we  obtain  the  values  of  m,  I  and  p  from  eq.(A.4)  as 


2A  cosh(^^ — —)  sinh(— ) 
77  77 

2Acosh(^^^t^)sinh(-) 
77  77 

2A  cosh(— )  sinh(— ). 

77  n 


(A.6) 


Thus, 


cosh(^) 

cosh(^) 

cosh(2:^) 

cosh(^) 


(A.7) 


which  gives 


Using  the  identity 


ml  cosh(^'^-^)  cosh(^^^^) 
cosh^(^) 


(A.8) 
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cosh(yl  -f  B)  cosh(^  —  B)  =  cosh^  A  +  cosh^  B  —  1, 


we  obtain  from  eq.(A.8) 


(A.9) 


nd  _  cosh^(^)  +  cosh^(^)  -  1 
cosh^(^) 

cosh^(l)  • 

Since  cosh^(^)  >  1,  it  is  clear  that  ^  >  1  for  the  Su  distribution. 
Applying  the  identit}^, 


(A.IO) 


cosh(  A  +  B)  +  cosh(  A  —  B)  =  2  cosh  A  cosh  B. 
to  the  sum  of  m/p  and  l/p  in  eq.(A.7),  we  get 


(A.ll) 


m  I 

- 1 — 

P  P 


2cosh(^)cosh(^) 

cosh(^) 

2r 

2  cosh( — ). 

V 


Solving  eq.(A.12)  for  p,  we  get 


^  cosh-M|(^  +  ^)] 


(A.12) 


(A.13) 


Since  p  >  0  is  assumed,  the  positive  value  of  the  inverse  of  cosh(.)  must  be  chosen. 
The  expression  for  cosh(2r/7/)  in  eq.(A.12)  can  be  substituted  in  eq.(A.lO)  to  give 


ml  _  cosh^(^)  +  -  1 

cosh^(^) 


(A.14) 


Solving  for  cosh^(7/7/), 


,2/7.  _  (m  +  l)^  -  4p^ 
4(m/-p2) 

Since  cosh^(A)  —  sinh^(A)  =  1,  eq.(A.15)  leads  to 


(A.15) 
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•  {m-lf 

sinh  (  )  = 

Tj  i{ml  —  p^) 

(A.16) 

Thus, 

^77^  2(m/-7)2)i/2- 

(A.17) 

Unlike  77,  the  parameter  7  may  be  either  positive  or  negative.  Thus, 

a  determination 

of  the  sign  of  the  numerator  in  eq.(A.17)  must  be  made.  Taking  the  difference  of  Ifp 

and  77i/p  in  eq.(A.7)  and  applying  the  hyperbolic  identity. 

cosh(A  B)  —  cosh(A  —  B)  =  2  sinh  A  sinh  B, 

(A.18) 

yields 

l-m  2sinh(7)sinh(Y) 

P  cosh(^) 

(A.19) 

Both  cosh(7/77)  >  0  and  sinh(2r/77)  >  0  (since  r  >  0)  and  hence  the  sign  of  sinh(7/77) 

is  the  same  as  that  of  1  —  m.  It  follows  that 

^7^  2(777/  -p2)l/2 

(A.20) 

and  7  is  given  by 

7-,s.nA 

(A.21) 

As  indicated  earlier,  the  expression  for  p  is  given  by  eq.(A.6)  as 

p  =  2A7  cosh(— )  sinh(— ). 

77  77 

(A.22) 

Sinh(r/77)  can  be  obtained  by  using  the  relationship  sinh^  A  = 
with  eq.(A.12).  This  gives 

—  in  conjunction 

•  12,  ^7  ,^“1.  m  +  l-2p 

*»M-)  =  (  2  >=  4p  ■ 

(A.23) 

cosh(7/77)  is  known  from  eq.(A.15).  Using  eqs.(A.22),  (A. 15),  and  (A. 23),  we  get 
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(A.24) 


Consequently, 


_  (m  +  /  -  2p)(m  +  I  + 

^  '  4[p(m./ — 


2p(^  -  1)^/2 

(7  +  ?-2)(7  +  ?  +  2)''"' 


Finally,  consider 


Pr  +  P-r  =  2e  —  A[sinh(-^^ - )  +  sinh("^^  )]. 


Using  the  standard  formula, 


(A.25) 


(A.26) 


sinh(A  +  B)  +  sinh(A  —  B)  =  2  sink  A  cosh  B, 


(A.27) 


eq.(A.26)  can  be  rewritten  as 


'y  T 

Qr  +  Q-t  =  2e  —  2Asinh(  — )  cosh(-). 

7]  p 


(A.28) 


Cosh(r/p)  in  eq.(A.28)  is  obtained  using  the  relation,  cosh  A  =  — ^ 
from  eq.(A.12),  that 


2  ^4  ^  i±cosht2^_  mows 


cosh^(— )  = 


r,  1  +  2p  +  m  + I 


(A.29) 


Thus,  using  the  values  of  sinh(7/p)  from  eq.(A.20),  cosh(r/p)  from  eq.(A.29)  and  A 
from  eq.(A.25),  it  is  seen  that 


,  _  9  2p[p(m/-p")]^/"  l-m  (2p  +  m  +  /)^/^ 

gr+9-r  -  ^[^^^;_2p)(^4./4.2p)l/2J4(^/_p2)l/2h  2pi/2 


p(/  —  m) 
m  +  I  —  2p 


(A.30) 


Consequently,  we  get 


-  7) 


e  -  .jlsr  +  9-r  +  T^TTiTTpl' 

“  ^  p  '  p 


(A.31) 
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A. 2  Johnson  Sb  Distribution 

The  transformation  of  the  Johnson  Sb  Distribution  is  of  the  form 


^  /  G-t  , 


(A.32) 


where  R  is  the  standard  normal  random  variable  and  e  is  a  location  parameter,  A  is  a 
scale  paramter,  and  7  and  7/  are  the  shape  parameters  of  the  Sb  Distribution.  Solving 
eq.(A.32)  for  G,  we  obtain 


G  —  c  + 


1  +  exp 


(A.33) 


As  was  done  in  the  previous  section,  we  now  proceed  to  find  m,  /  and  p  in  terms  of 
the  parameters  7,  A,  7,  and  e.  Thus, 


m  =  gzr-  9r 

A 


l+exp(lf^)  l+exp(lf)' 


(A.34) 


Therefore, 


exp(2^)  —  exp(^4r^) 

"*  “  1  +  exp(  +  exp(  +  exp(  2:^)' 


and  consequently. 


m  =  \ 


exp(^)exp(-j^)[exp(^)  -exp(7;=^)] 

1  +  exp(  +  exp(  +  exp(  22^) ' 


This  implies 


_  ^ _ 2exp(y)sinh(^) _ 

exp(^)[exp(-2:^)  +  exp(^)  +  exp(^)  +  exp(2^)] 


Finally,  we  get 


(A.35) 


(A.36) 


(A.37) 


^  cosh(2^)  +  cosh(n' 


Proceeding  in  a  similiar  fashion,  we  find 


(A.38) 
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P  =  9r-  g-r 

.  sinh(^) 

cosh(^)  +  cosh(^)' 

In  summary,  the  values  of  tnj  and  p  are 

^sinh(^) 

cosh(^)  +  cosh(^) 

cosh(^)  +  cosh(^) 
-^sinh(^) 

cosh(^)  +  cosh(^)' 

Hence, 

cosh(^)  +  cosh(^) 
cosh(^)  +  cosh(^) 
cosh(^)  +  cosh(^) 
cosh(^)  +  cosh(^) 

which  gives 

P.E  -  — 

ml  B 

where 


m  = 

/  = 

p  = 


(A.40) 


(A  .41) 


(A.42) 


(A.43) 
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(A.44) 


A  =  cosh2(-)  +  cosh(^^^-^)cosh(^^^-^) 

1]  T]  7J 

+  cosh(-)  cosh(^^^-^)  +  cosh(-)  cosh(^^^t^) 
V  V  V  V 


B  =  cosh^(— )  4-  cosh^(— )  +  2cosh(— )  cosh(— ). 

n  n  n  V 


Using  the  identities 


(A.45) 


cosh(A  +  B)  cosh(A  —  B) 
cosh(A  +  5)  +  cosh(A  —  B) 
cosh^(A)  —  sinh^(A) 


=  cosh^(A)  +  cosh^(B)  —  1 
=  2  cosh(A)  cosh(B) 


(A.46) 


and  applying  them  to  eq.(A.45),  we  get 


A  =  cosh^(-)  4- cosh^(— ) 

1  j; 


4-  2  cosh(— )  cosh(— )  cosh(— )  4-  sinh^( — ). 
V  V  n  V 


(A.47) 


B  is  smaller  than  A  because  the  cosh  function  assumes  only  positive  values  and  since 
cosh(^)  >  1  and  sinh^(^)  >  0.  This  gives  ^  >  1  or  equivalently,  ^  <  1. 

From  eq.(A.42) 


^  p_  _  2cosh(^)  4-  cosh(  ^)  +  cosh(^) 


cosh(M  +  cosh(^) 


(A.48) 


P  _  2cosh(^)4-cosh(^)4-cosh(^) 


I  cosh(^)  4- cosh(^) 

Multiplying  the  expressions  in  eq.(A.48)  and  using  the  identities  in  eq.(A.46)  we  get 
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(A.49) 


where 


C  —  4cosh^(— )  +  2cosh^(— )(1  +  cosh(— ) 

V  V  V 

+  4  cosh(— )  cosh(— )(cosh( — )  +  1)  +  cosh^( — )  —  1 
V  V  V  V 


(A.50) 


D  —  cosh^(  — )  +  cosh^(  — )  +  2  cosh(  — )  cosh(  — ). 

V  ;;  1/  7/ 

Using  the  property,  cosh(2A)  +  1  =  2cosh^  A  in  eq.(A.50),  we  get 


(A.51) 


C  =  4cosh^(-) +4cosh^(— )cosh^(-) 

T]  r]  Tj 

+  8  cosh(  — )  cosh^f-)  +  4  cosh'‘(-)  —  4  cosh^(-). 
V  V  V  V 

Thus 


(A.52) 


C  =  4cosh2(-)T». 


Finally,  from  eqs.(A.49)  and  (A. 53)  we  get 


(1  +  — )(1  +  y)  =  4cosh^(-). 
ml  V 


Solution  for  t]  results  in 


(A.53) 


(A.54) 


”  cosh-lH(l  +  i)(l  +  f)}'7^r 

Now  consider  the  sum  of  the  terms  in  eq.(A.42).  Using  the  identities  in  eq.(A.46), 
on  this  sum  we  get 
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(A.56) 


P_  P  _  2cosh(^)  +  cosh(^)  +  cosh(- 
m  "*”  /  cosh(|)  +  cosh(^) 

_  2cosh(|)  +  2cosh(^)cosh(^) 
cosh(^)  +  cosh(^) 

Solving  the  above  equation  for  cosh(7/7;),  we  get 

.  cosh(-)e  +  f  -  2) 

cosni— )  =  - - = - 

1  (^  +  f  +  2)-4cosh^(:) 


(A.57) 


where  again  we  have  made  use  of  the  identities  in  eq.(A.46).  From  eq.(A.54)  expres¬ 
sions  of  cosh(r/7/)  and  sinh^(r/7/)  are  obtained  and  substituted  in  eq.(A.57).  The 
result  is 


cosh(— ) 


2(1  +  m  +  f  +  mf  -  m  “  f  “  2) 


(A.58) 


2(^-1) 


Rather  than  solve  this  for  7,  it  is  preferable  to  derive  sinh(7/7;)  because  sinh”^(.) 
yields  the  correct  sign  of  7  due  to  it  being  single  valued.  Again  we  use  the  identity 
in  eq.(A.46)  to  get  sinh(.)  from  cosh(.).  Thus 


sinhfi)  = 


(A.59) 


To  see  which  root  should  be  taken  in  the  first  factor  in  the  numerator,  observe  from 
eq.(A.42)  that 


p  p  2sinh(l)sinh(y) 

I  m  cosh(^)  +  cosh(^)  ’  ^  ’ 

where  the  hyperbolic  identity,  cosh{A  +  B)-cosh{A-B)  =  2  sinh(A)  sinh(.S),  is  used. 
The  denominator  on  the  right  of  eq.(A.60)  is  always  positive.  Also,  sinh(2r///)  >  0 
since  r  >  0.  Hence,  it  follows  that  the  sign  of  sinh(7/77)  is  the  same  as  that  of 
p/l  —  p/m.  Eq.(A.59)  then  becomes 
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(A.61) 


The  value  of  7  is  consequently  given  by 


7-7sinn  [  2(^f  -  1) 

The  parameter  A  can  be  evaluated  using  the  expression  for  p  in  eq.(A.41). 


(A.62) 


p[cosh(^)  +  cosh(^)] 


(A.63) 


sinh(|)  ^  ^ 

Cosh(r/77)  and  sinh(r/77)  are  known  in  terms  of  p,  /  and  m  from  eq.(A.54)  and 
cosh(7/p)  is  obtained  from  eq.(A.59).  Thus 


A  =  p- 


{1(1 +  ^)(i +  ?)]■'"  + 


((i  +  £)(i  +  f)-'‘)‘i" 


Going  through  the  algebra  we  get 


i(i  +  ^)(i  +  f)]'''ii  +  #^i 

_ V  ml 

((l  +  i)(l  +  f)-4]>fl 


A  =p 


which  gives 


.  [(i  +  i)(i  +  f)r7(i  +  a(i  +  f)-'‘i''" 

A  —  p - 5 - . 

(S-i) 


Consequently, 


A  =  p 


{[(1  +  a(n-f)p-4(i  +  a(i  +  f)}‘^^ 


(fe  “  1) 


Finally  the  formula  for  A  is  obtained  as 


(A.64) 


(A.65) 


(A.66) 


(A.67) 


,  {l(i  +  :^)(i  +  f)-2P-4}'i^ 

(S-1) 

For  e,  eq.(A.33)  is  used  to  determine  the  sum 


(A.68) 
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Qr  “I"  9—r  —  26 


1  1  , 
+  ll+exp(lf)  +  l  +  exp(lf)‘ 


This  is  easily  shown  to  be  equivalent  to 


sinh(^) 

9t  +  9-r  =  26  +  A[1 - ,  /2\  I  ''  1  /rJ- 

cosh(^)  +  cosh(-) 


From  eq.(A.63),  this  reduces  to 


,  Psinh(^) 

9t  +  9-r  —  2e  +  A - .  ■ 

smh(-) 


(A.69) 


(A.70) 


(A.71) 


Substituting  the  previously  determined  values  of  sinh(7/77)  and  sinh(r/77)  in  terms  of 
p,  /  and  m  from  eqs.(A.61)  and  (A. 54)  respectively,  quickly  yields  the  desired  result 
for  6  as 


9r+9-r  A  p(f  - 

A. 3  Johnson  Sl  Distribution 

The  Johnson  Si  Distribution  is  given  by  the  transformation  of  the  form 


(A.72) 


R  =  +  rj  in(G'  —  e) 


(A.73) 


where  again  R  is  a  standard  normal  variable,  e  is  a  location  parameter,  and  7  and  77 
are  the  shape  parameters.  Solving  eq.(A.73)  for  G  we  get 


R  —  'y* 

G  =  6  +  exp( - ). 

V 


(A.74) 


Then 


^  —  PSr  9r 

=  exp(^^^ — —)  -  exp{- — —) 

7}  TJ 
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/  =  g-r-  g-3r 

,-r--y\  ,-3r-7% 

=  exp( - )-exp( - ) 


V 


V 


P  —  9t  g-T 


^  ,-r-7  , 

=  exp( - )-exp( - ). 

n  P 


It  follows  that 


m 

P 

l_ 

P 


2r 

exp(— ) 

P 

-2r 

exp( - ) 

P 


and  therefore  ^  =  1. 


Moreover,  the  value  of  g  is  obtained  from  eq.(A.76)  as 


2r 


P  = 


From  eqs.(A.75)  and  (A. 76), 


which  yields  7*  as 


Finally, 


p  =  exp(-£)|(!^7^-(V) 
P  P  P 


m  _  2 


'y  T  T 

Sr  +  g-r  =  2c  +  exp( - )(exp(-)  +  exp( - )]. 

n  n  s 


(A.75) 


(A.76) 


(A.77) 


(A.78) 


(A.79) 


(A.80) 


Substituting  the  known  expressions  for  exp(— 7*777)  and  exp{r/g)  from  eqs.(A.75)  and 
(A.76)  respectively,  we  get  the  desired  result  for  e  as 


e  = 


gr+g-r  P7-  +  1 


2  22.  _  1' 
P 


(A.81) 
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Appendix  B 

Connections  between  ga,  ka,  Pa 


According  to  eq.(48)  of  chapter  2,  the  Gaussian  random  variable  R  is  related  to  the 
non-Gaussian  random  variable  G  by  the  transformation 

R  =  ^  +  vMG-,X,e)  (B.l) 

where  fi{g‘,X,e)  are  single  valued  monotonically  increasing  functions,  i  =  1,2,3.  Let 
To  and  go  satisfy  eq.(B.l).  Because  of  the  single-valued  monotonically  increasing 
nature  of  the  transformation 


Pr{G  <  go)  =  Pr{R  <  vo).  (B.2) 

From  a  relative  frequency  point  of  view 

„  ,  No.  of  observations  less  than  equalto  ro 

Pr{R  <  ro)  « - - - - - - -  (B.3) 

n 

where  n  is  the  total  number  of  observations. 

Now  assume  that  n  observations  of  the  random  variable  G  are  obtained.  Ordering 
the  observations  of  G  from  the  smallest  to  the  largest,  denote  the  ordered  ob¬ 
servation  by  g^.  Then  k  equals  the  number  of  observations  less  than  or  equal  to  g^. 
Corresponding  to  the  ordered  observations  of  the  random  variable  G  are  ordered  real¬ 
izations  of  the  random  variable  R  (See  eq.(2.18)).  Denote  the  k^*^  ordered  realization 
of  R  by  r^.  Because  the  transformation  in  eq.(B.l)  is  single  valued  and  monotonically 
increasing,  it  follows  that 
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Pr{R  <  r^)  =  Pr{G  <  5^)  «  ^  (B.4) 

Introducing  the  “continuity  correction”,  as  was  done  with  the  q-q  and  p-p  plots  in 
chapter  1,  Pr{R  <  r^)  is  approximated  by 

k  —  - 

Pr{R  <  r^)  w - 2..  (B.5) 

n 

By  definition, 


Pa  =  Pr{R  <  a) 

where  a  =  3r,  r,  — r,  — 3r.  We  define  the  integer  ka  such  that 


where 


(B.6) 

(B.7) 


ka  =  InPa  +  ^],  (B.8) 

[.]  denotes  the  closest  integer  and  a  =  3r,  r,  — r,  —  3r. 

Equations  (B.6)  and  (B.7)  imply  that  a  approximately  equals  the  k^^  ordered  sam¬ 
ple  of  R.  Note  that  the  k^^  ordered  sample  of  G  is  where  r*“  and  satisfy 
eq.(B.l).  From  equations  (B.4)  and  (B.5) 

k  —  ~ 

Pr{G  <  /“)  =  Pr{R  <  /■“)  «  Pr(R  <  a)  «  ^ - 2..  (B.9) 

n 

It  follows,  given  Pa,  that  one  can  determine  ka,  and  given  ka,  one  can  determine  ga 
by  the  simple  relation 


(B.IO) 
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